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This report describes a stqdy of the teachers and students in the * 

West- New York Adult Learning Center. The purpose, of the study was to find 

out which patterns of classroom .interaction and -students' characteristics 

Were most highly related tp the acquisition of oral proficiency in. English 

by adults who were learning English as ^second language. *' 
^ * ■ . ' '* I 

The report is written in two ways, TJife major portion. Chapters j 

Three and Four, are written primarily for those interested in the details 

of the statistical "analysis. Chapters One, Two, the first part- of Chapter 

1 

Three, and Chapter Five have been written so that the reader, who does not 
have a statistical background may understand the methodology and results 



of the study. Chapter Eiye is in fact a summary of the entire studjr wi 



hout 



the detailed explanatlo'ns of -the methods presen^d* particularly in .Chapters 



Three and Four. Chapters One ftiid Five were written by the Project Director, 
Chapter Two and the description, of the Observation System In Chapter Three 



by Meredith/ Stone, Associate .Project Directpi-, and the analysis of the,^ 

'ft. ^ , . " . ' 

observational data in Chapter Three and Chapter Four by Allen Yates, / 

Research Statistician*. ^4 , . • . / , 

\* ^ . ^ ^ i i I 
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CHAPTER ONE 



THE DESIGN OF THE STUDY 



• % 



- 3 - 
INTRODUCTION 



This report describes. the work conducted over one year in the West New ^ 
Yorlc Adult Learning Center to detepni'ne .the relation l)e'tween teaching st^ate^xea, 
methodologies, and perfomances^^^-^^^X^cquisitig^^^ facility by the students 
at the^'Center in speaking English as a s^cou(j[^ang.uage. The We§t New Yotk Adult 
Learning Center provides a training prog^^ in whiah adults learn to speak^ 
English as a second language* The adults attend classes;either during the day . 
or in the evenings. Instructi9n is provided at thre,e 'different levels adapted 
to the proficiency in English which this students have already acquired, ' 

The purpose of this jproject was to determine which kinds of teaching per- • 
f ormances (sometimes called competencies) were diirfectly associated with. t 
, differences in acquired proficiency in English on the part of the .students. *^ 
The methc^dology used in the study related differences in teaching styles and* \ 

pBTf ormances ataong.the teachers to differences in the acquired proficiency of ^> 

< . * ^ ^ ^ ^ • f • * 

their- students. ' • . 

%^ ■ " / ■ - • . • 

lAc^uired proficiency, in English. in this case meant acquisition of the 
ability to speak English. The Center does not attempt., except indirectly, to 
improve students^ abilities^.to write or read English. The instruction is 
directed primarily to stimulate acquisition of facility in speaking English, 
and the. procedure/ used by the teachetjs rely heavily on 'oral discourse between 
teachers and students. The students themselves vary considerably in age, 
previous. education^ and previous experience with English. ^ 

^ ^ In instructing the ^students the teachers use^a^^yariety of technicjues 
and methods, though t\^o methods, Cthe audio-lingual and the "silent way," are 
the principal methods used. Some. teachers adhere to orfe.method; others. choose • 
among methods. . • • - . . . 



1^ ' .. -15 



Relatively little fprmal research has been done on the ef f icac^'^o^t , 
^ . ' /' ■ -. ' . . : • . , . . 

efficiency of specif;Lc teaching procedures, though, there are advbfeates pf one ' 

■/ ! " ■/ ■ ' ■ . 

or another method. The approach taken fn this study was to look at all prQcedures 



used by the teachers /anc|;4tteinpt to identify those which, irrespective of method, 
. were most highly related to.jgains in acquired proficiency in speaking Englisli.y 



4 - ^^3t, is conceivable that some teaching performances are highly effective irrespective 
of the general method in which \they/are embedded. Or, it may be that a cluster 
of teaching performances characteristic of one method may be more effective , 
. than those of another method. A thir4 possibility is that some procedures or 
* ipethods may be more effective at one ^tage^ in the learner's acquisition of 
, speaking proficiency, and others ate more effective later stages. 

/ , ^ , The general methodology used in' the study (whl,ch will Ije descrlhed in 

greater detail in the following pages) r^quireid us. to measure student proficiency 
at two points in time, and to observe daily' the intervening instruction; ^ ^ 
The measures tt'i speaking facilitA^^ere either direct measures of proficienc/, 
sxich *as 'the Oral Ptoficiency-TestV^eveloped Specifically, for this project, or 
,-.^were othei: measures of knowledge of English. ,The observations provided a ^ 
^continuous record of both -teacher and student behavior during clas§ senssions and 
were made between the two points of assessment of the students' speaking 
proficiencfy* , * * ' . \ 



CONCEPTUAL BASIS OF Tte METHODOLOGY - - « t 

The methodology used in this study is built on tWo major hypotheses and a 
.few minor assumptions. The first* of these hypotheses is that among a grojup of 
^eachers of English as a second language are some who are jnore effective in 



ERIC- , - , • /fb 



"producing speakittg^^r^f-iGl^nay^ii^ This hypothesis does 

^ V, ' ' • — - - 

< <^ , ' 

not imply that' the -less effectiye teachers^ are necessarily incompetent or. have 

^ • >^ • ■ ' ' ' ' ^ 

no success in facilitating the accjuisition of English as a second langpage. 
,It does imply^that some* teaqjiers are more skilled than others, in* helping their 
students make greater ^ains inacquiring the. ability to speak English. prof ipiently^ 
. ^ This hypothesis i-& an- old idea — in ^y profession o^ qraf t thete seems to 
be practitioners who are recognizabj-y more effective than other pi^ctdtioners. 
These more effective, practitioners presumably possess skills which account *^^r 
their relatively greater efficacy. The purpose of the methodology is to ide^ify^ 
.these mote, effective practitioner^ and to identify what it is t*hat they do \ 
which accounts for their greater success in producing. prof iciency in speaking 

English/ ' ' . / * • ! ' i- . • . 

The second hypothesis was Uhat a teacher's performance^ that is^ what the \ 
teacher does, ^accounts for the acquisition Uf prof icienc^ .i Teachei?^^^ 
queistioiis^ provide models *of desired, ^correct, or' appropriate respons.es, and 
indicate to the. student when he^ or she has made an appropriate or an inappro- * - 
. priate. response in speaking.. These and similar auctions of the teacher elicit 
. the. desired speaking r^ponses, shape or modify them, or . stimulate the processes^ 
by whi<ih the^jbudents attempt to generate English statements. - ^ ^ 

The teachps' attitudes tbward the students ^nd Jtoyar^ the language b^eirjiS, 

t ■ \ * I ' 

learned and toward , the language and culture of the .students are undoubtedly 

important facty&rs in stimulating the students ta tryi t4 learn English and in 

tfieir responsiveness to instruction. Differences among the teachers in these 

characteristics are small and probably would be unr^T:^t^d ' to differences in :the 

students' l^atning. The teachers in the Center are carefully selected for 

this work; they work closely together; and the atmosphere^ of the Center 16 warm, 

friendly*and supportive. The staff knows. the students well and acco^miodate^ 




V 



instructign tcr their needs. 



\ 
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But the .actual instruction is mediated by the kinds of p^rtformances ihV 

' • ' ' " / - ' /' ^ 

which the teacher speaks English and^ helps the student to speak ^English, t 

, r ' / ' ♦ * ' 

It is, therefore 2' Vi^esumed that t:he strongest relations between what .the* teach^^r — 

" V V ' / ' ' ' ^" ' ' ' ' ' 

does ^nd student ifeatning will be found by looking a^t those perfonpances which 

' ^ / . • ' 

are directly instructional in /character. ' . . ^ * — 

' Differences in the characteristics of the students may also accoufit for 

differences in their acqui^red proficiency. Some may ha ve more Aptitude for 

. / - . - '• 

learning languages. Or /the differences in their achievement of proficiency may 

'I ' ■ 

b4 i^elated to. their previous experience with English or C^ieir educational attain- 
ments or their socioj^cpnomic background and* current status. , The competing 
hypotlfe^s to tlie one th^t attributes differences in prof £ciencj>/to differences 
'^in teachdM styles is that it is such differences as these among the students 

^- which* accoi^^ tor their differences in learning. ' \ ' * 

" / ' ' ' ' ' • ' : 

In this |Stuay information was gathered on a Variety of the students' ^ 

chara^tfe^fli^ti^ which were statistically related to dlfferenpes in proficiency in, 

^speaking Englj^ 'achieved by the end of the school year.^ This study tshus provides, 

data oqlf the degree to which specific teaching performances' ar combinations of ^ 

them acjicoujit fb.j^^i^f erences in st^ents* Acquired proficiency and on the degree 

^ .to whicti d|.f!jFerences among the students themselves account for thei,r acquired 

proficiencies. 



/ 




.Finely, the assumpt:^n was made that correlationSi between teacher., performances 
^d gaina^ in attfdent leai^ing are a reasonable basis for ascertaining which 

ormanocs/^re ^ost likely to influence ^tudentsj acquisition of - ^ 
EpglishX?\ ^\ ssco'^^ l^^S^g^ ^^^'^ awar^ that correlation does not 
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necessarily imply causation but we aiie.also aware that where correlations 
are foimd causal Connections of some, kind may be, inferred, subject/to verifi- 
cation by careful experimental study. This study, therefore, ^as designed to 
' find out if such correlations existed and their relative size, and from such 
correlations to conceptualize a* hypothetical ^picture of teaching performance 
likely to be associated with greater gains in 'acquiring, proficiency to speak 
. Bnglish as a second language* \ ' 

> Of the two hypotheses .mentioned above, the first is the more tenuous. 
Miile> it seems reasonable to hypothesize that there are varying d^g^ees of 
teaching effectiveness, it is possible that in selecting. a particular sample 
of teachers one may pot find diffe'r^ces among the teachers either in the 
effects .they produce^jt>r-i:5Lj;l^^ skills. Thus, a study of this kind ' . 

inevitably runs a/risk that the particular skmple of teachers involved ^ill 
not produce the necessary inforroatioh. But praetipAl experience suggests that . 
- in -almost any grqup of teachers one can find teathers who are more effective 
„ / then other teachers, apd it was on the basis of this practical experience 
^i^that this study was conducted. 
'.A'' The second hypothesis is a theoretical one.' It postulates ihat the learner 
' directly affected by those stimuli which impinge on him or her. But the 

■ learner processes these ,stimu;Li in some way so it is -equally reasonable to 
V 'assume that chajract^risties. of the learner are also^ directly related to how much 
- the learner achieves. • For the^latter reason, ' we have fathered as much Unformat ion 
as was^possible in the context of this study about -the ' characteristics>f the 
learner to determine the exttent to which" these, characteristics might, influence 

learning prior to and,iiiidepeh|dcnt of any influence of teacher performartce. 

.' ' ■ ' • ^- •;' • ■ •' • ' . - •■ < ' ■■''■,>■.'- 

.y \.- ' • •• 

. - " . ' '-' • • ^ '-i^^^i -'\ 
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It is conceivable that some teaching, performances in conjunction with tertalt) 
characteristics* of students will provide either highly effective or relatively 
ineffective combinations related to the amount o£ learning achieved by the 
.students. " . ^ 

The final assumption is*no more thatj a. statement of what is implied by 
finding ^n association/ or correlation between two -sets of events. One can, 
neither* draw strict^ inferences about a causal relation between the events noj: 
can one dismiss the possibility of one. The analytic methods used iix this study 
examined the ^hypothesis that teaching performance must be* taken into account^^ 
in order to better predict student learnifTg, with the apparent implication 
^ that teacher performance influences how much, students learn.. .However, experi- 
mehtal studies have to be done to test the validity of any caUsal hypotheses 
derived from a c^orrelational study such as this. - /\ , • * 

These hypotheses .and a^suniptions outlined above are the underpinning of the 
' methodology. The essence of, the methodology is to relate v^iation in teacHlrig 
performance to differences in the degree to which students achieve proficien.cy. 
If correlations are found between measures of teaching performance' and measures 
Of student proficiency, such relations become the basis for establishing 
hypotheses about causal connections between teaching perfbi;mance and student 
learning. ' ^ ■ ' • . 

The^above paragraphs suggest the limitations-of this study. as well as its 
t potential value. ' best , from a study of the type conducted, one can exatnine 
tthe hjrpothesis that certaiir-types of teachers'^ actions or activities make a ^ 
-difference, in Jihe accuracy with which we can predict* students' learning. The 
data reported do not represent proof that there is a direct connection. On6 ^ 



• '4 r , ■ ■ 

y ^ - ^ . . • . ^ . , • . * ' r. > , 

can also /estimate how much of a role any ^fven .teaching pejrformante ^ilays 
'in the predictipn of Wtude^nts' learning, but the >ame cautiQTi in making. 
: causal* inferences applies here also. 

Th6 multivariate statistical methods used in the study are appropriate • 
for data of the kind gathered. Pactor analysis and canonical discriijinant 
function analysis .were used to reduce both, student-performance data and , 
teacher-performance, data to feheir underlying dimension/ Multiple jregression, 
canonical •cori;elation, and factor analytic methods were used to relate the '\ , 
■ dimensions of, teacher perfolrmance to those of. ^tud^nt-^perfofaance. In the - 
.followifig chapters the bfeic jescriptive an«l correlatioijal -data^among a variety 
of student performance and^teadfer performance variables are presented^.. The 
a*lysis then 'moves -on 'to multivaria.te- studies- of classroom 1^^ It 
• ' con^lu^s with an analysis of_how Student achievement"ahd classroom. interaction 

■ - ^ ' ' ■ f . • . . ■ - . 

: variables 'relate to each, other. In these ^nalysTes a'ccount is taken' 'ojf students' 

^ . • ». 

^background chawcterisUcs and their proficiency in English at the beginning ■ 

■ ' ' , ' ■ , ■ ^ 
of the data-gatheriiiis- phase of, this study.- • ^. 




LG OF PROFICIENCY IN 'SPEAKING ENGLISH ' . 



. There are ob^ipusly degrees of proficiency in s^^aking English, even among ^ 
native speakers. Adults learning .Engli^ as a second language wil^ also acquire 
"'skill in the language to varying ' degrees . But' what is meant ."<ftfferent; degrees 
- • of proficiency"? The mekning->-f -proficiency neeis_^'be operational^' defined" 

before undertaking a study of the^-'Mnd described here. . ^ ■ 

Hr • • • • 

■ The dperational definitioji of proficiency used in this study was' derived 'by- 

• - ■ . ■ ■>•..-.■ 
« , 'an analysis o'f the goals of . the West W York Adult .Learning^Cen^er.^ Thes€ 

.. ■: goals ar^ to facilitate the acquisition of English as a second language so that': 

■ ' \ . ' * ' . • : ' 

ERIC . ,. , \ ' 2i. , . ' , 
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(1) the, adult learners can understand -conversational English; (2) they will 



be able to communifcata in English in ordinary situations so that they are 

• ■ • t . • ■■ " . - • 

adequately nandpr stood; and (3) they wilj. acquire the= bSsi^ structures 

u , ' * • « 

the language SO that they are likely to continue to gr(3\<. in proficiency. 

. ,The meaning of^ these' goals is best understood by* thinking about how the 
learners who come to this Center .will use English. These adults are immigrants 

to. the United States who have been in this couAtry varying amounts of time. ] . 

> lb ' ' ' ^ } ' , ■ * 

The majority of them have, lived in a community wKere speaking Spanish is the^. 

notm. Most," but not all students at the Center, come from Spanish-speaking 

"countries, principally Cuba. They wish to acquire sufficient pr<J^iciency in 

English so thai: th^y can communicate with rion-SpanisJi speaking people in stores, ^ ^ 

ia clinics and hospitals, j.n schools in places of emplo3nn^t,, arid, in interactions 

with- gQvernm4'tit agencies, of one kind "Or anothfr.^ In ojihex words, they wl$h,to 



^beco^^, and the goaJ^ of the school is to help them become, functionally proficient. 



ijci Speaking English^ The aim is to .h^lp them a'^h^eve ?uf ficltent facility in En'gJ.ish. 

i ' ' • ^ - . • '* \ - V- 

sa that they are in .fact bilingual fot the pt;actical purposes of everyday living. 

Three levers of performanqe in' speaking English related tp this goal u»ay be* 
distinguished. The fii^st level is , acquisition of V:he larfguage, such that the 
person understands ordinary communifcatipns to TieT or him; for e^tample, a* person Is 
asked simple que^stions such as occur in, everyday conyersa4:ion and is -able to 
understand the question being asked even though tfiey cannot always provide a fuil« 
or accurate answer. A person Who h^s attained this , Ip^el^f prctficiency Is able 
to ^understand ii\ps*t simple communications, but cannot Respond adequately and with 
facility. . 

A second level of proficiency is repi^esented by the learner^ being aSle to 

' ft . ' . 

^ respond to questions or to make statements \about^himsifelf ^oif herself, wifet they 

\ ERIC \ ■ 2^ 




plan to do, what they think on practical matters, and the like. To'N^gOTQunicate 

at this Ifevel, a* person must have acquired the basic structures of tiie English 

language. * * ' * ^ ' 

/ • . - , ' 

. ' '.A person has attained. the third level of proficiency when he or she can 

generate questions and statements on his or, her own, can extend .discourse through 

a series of statements or questions, and in speaking uses mojre complex structures. 



Eajch o;f tlfese levels 6f proficiency may have^one-bt ulDre^of X>iree 
characteristics: (1) the person fepoken to may-give evidence of ^ understanding - 
. ^ the language spoken to him but does^ not respond with facility or accuracy or 
'*complet;eness; (2) .the person may both understand and use\appropriate stru;:tures 
VbVt may make errors in the use of th^ language; (3) th^ petson may both under- ^ ^- 

stand and respond with, appropriate, st;ructutes and them accurately.. « 
; * T^s, there are t^o underlying concepts by which proficiency has^ been ^ 

' described^ bne of tliese concep.ts describes the level of- language usage avail- 

,-\ . * ') . ^ . V , , 

able .to the individual. On this dimension performance ranges from sufficient ^ 

usage^o comprehend what i^ being heard to the ability to generate relatively 

, ' ' ' ' * / * ^ . ' ♦ 

complex structures in ektend^d d^iscourse. , ^. 

the other .dimension is that of the accura<^l^f the form of the communication; 

Accuracy means tBat a' person uses 'English sentences which, jare structurally correct 

J* and (by implicatibrf); has also used word^ correctly. * .* ' 

The West New York Adult Learning Center defines the kinds of structures, that 
• *• • • " * • , ^ 

are to be a^cquir^d at^.each level of .instruction. ThQ acquisition^^aml qse of these 

structures defines operationally' what is meant by proficiency. Proficiency means 

^ / , ' • . - . . 

faciltty and accuraqy iifl the use of these structures. ' . x, * 



• ; The Oral Proficiency Test 

, , , ^ * \ . - 

An important aspect of this s^udy was .the development of •an oral proficiency 

test which w^s administered to students at the Center in thet spriug of 1976. This 
' , * ' • .y • ^ ^ . ' ' V 

test /which ^ill be describecl in ^ later '"^chapter) -utilized .*three formal for 

♦ * ' ' ' ' * ' « • 

eliciting t^e speaking of English.' In one part of the test tKe "fexaminer asked' the 

student quebtions abodt himself or herself to which the student may respond. The 

putpQse^^of* this procWur^ was to see if the' student could comprehend tfl|^^ question 

or statfeBient, and could respond ^appropriately. In the'iecond part of the test 

the students were presented with pictures of two, eyents and asked to describe whatr 

was happening. The purpose of tbis procedure was to estimate how well students 

^' i- • ^' . • » , • • ^ 

could ^generate language freely. I^i the third part the l^earn^ wa,s pr-esented 

^ • r • \' ' ; : • * i , . \ 

with cartoons from a Spanish newspaper .and was asked,>to explain' the cartoon. The 

* ^ '-;/ ' • '% ^ * ' ^ ; . ' ' ' ♦ * . ' 

purpose o£ tl}i$ procedure was to estimate how easily the student could move from 
idiomatic Spanish to equivalent Ei^glish statements. We assume that spart of t)ae 



process of acquiring language, facility is to acquire the ability to transfora 

/* ^ ' . 

concepts expressed in one language into equivalent concepts expressed in a second 

' ' ' r'^ ' * * . ' ' • ^ l\ 

language. This third assessment * format waa designed *to estimate how^ll these 

' ' * ' 

students could make such transformations. • ' ^ 

^ The students'' res][)onses were scored on three characteristics: (1)' comf)re- 
herision; {2) use o$ structure;^ an4 (3)' correctness. Comprehension meant that * 
the student showed by the response^ that he or she understood the questibn but 

may have or may not have responded accurately. Use of structure meant that ,in 

/ . ^ 

responding the student Used the appropriate semantijc and syntactical -forms of 
Bnglish but may qr may not have made errofs in the other parts of theNresponse. 
Correctness meant that thfe student s'poke one or more .English sentences which 



were appropriate and^wefre gTf^imnatit:ally accurate in every respect. Score's on 
these ^dimensions !lfor each student were the indicators of achieved proficiency 
in speaking English. 

Other- Measur.es of the ijffefcts of Insiferyction 



Although the goal of the A4ult Learning* Cenfer is to develop functional 

' * ' ' * I * 

proficiency ih speaking English, other-development* in the use- of language may 

occur as a conseqi/ence of the instruction Jjrovided.^ Th^ Center* gives .relatively 
•** * * ' 

litt'te^ttention -to formal, instruction in,reading, and. then mainly at tf{ie highest^ ^ 

^ \ '1 • . . / " • • 

leve^l of instruction. ll^ is possible ttiat acquiring f^cilit^ in spiking the 

language, seeing EnglJLsh words written on the board, redding papers and magazines 

and being al>le to decode the words bfe'cause of the language instruction,- the 

learner may acquire greater- facility /in reading ^he language. " . ^ 

We, therefore, used a functionall reading literacy test , as one^ measure of ^ 

increased language facility. A sed .of items had been developed ^n. another 

'■ * " ^ ' li ' ' ' ' ' 

prcject that measured the ability^ftf English-speaking adults to perfc^rm functional 

7v/" . ' * ' - ' I 

. It' 

reading tasks. .A fiinctional veaditig task, for e^^ample, "is reading labels of 
bottles, reading signs and dire^Wons, reading' instructions on forms and similar 
ta^ks which one meets in everyday life that require &ome reading skill. This 
test was administered to the ^tuflents at the Center twi^e in the year, once - 

r • / T' ' ^ ' , • , ^ ' / ' 

as 'a pretest and again as a^^ftgsttest, and changes in performance were calculated.'^ 
The purpose of this analysiW was to determine if the learners Hbd improved their 
abdlity to' read as a benef /ciaL Side /effect o£ their improvement in speaking \ 
English- '* " ,/ 



Another measure, a dect)ding test, was used on the presumption that one aspect 
'of acquiring language, proficiency is to acquire decoding skills in th^ language 



being learned. This ^ec(l)din| test had h^n Osed^in the tpast to nieaisure' the " * 

decoding skUls of English-speaking .children but is <^sable ^ith adults because 

^a speaKer of another language leirning to speak Etrglishrhas fo acquire the same 
• _ ' * \ *** « , * 

* set. of skills, ejjen though they ^may acquire them in a-'diSferent way. Since the' ^ 
graphemes, of English and 'Spanish are highly similar, the adult learning. Errglish 
i is acquiring new .or different' phoneme-grapheme^correspondences. The teachers 
aay give some attention to these dif fer^ences to .irfprove understanding and 
^--^Bronurici^tA'on;^ for the student^.inay detect thenf on their own. In either qase'' 

we Would ^expect. >6me improvement skill. ' The decoding test wks^also^^, 

\ ' " ^^^^ 

- administered W a pre- and po.st test and estimates wete made 'of the .amount '^of'-chjati^l 

Iruthis perfonnalTce skill, r ' - * ' 

We ^•presumed t^git one of the cbfisdquejdces of constant. -e^^posure to the phonology 

of English would be an increase in theSe 'skills. We also ^pre,5umed: that any 



increase in decoding skills .was a beneficial coris^cyien^e because the learner, 
will have, acquired a set 'of skills that can be transferred to a variety^of 
situations and whtfch should help him or her continue to develop proficiency in 
• «l&peaking English. . v " ' / , * 



Two other measures'' of proficiency were alsov^sed, the John -Test 'iilnd the 
*Morano Test. The Center had. been using these tesrs for several years t(^ estimate 
theijj^tudents* ^language proficiency. TheV were adri^istered as pretests fib, this 



study atid scorea^on them were correlated with scores on othfer tests. > * 
J There v^re two reasons for using the John and Morano Tests. . One of these 



: was psychometric: ,to obtain information on the reliability and validijty.of 
/\:hese tests. Such ^information is necessary to 'determine .whether or not the, 
procedures used to^ssign students to levels of instruction^ are accurate andv * 



effective. The other reaspn was" that the set of five measures used in ;the - / 
study, (the Johrf Tes£, the koraoQ Teslt, the Oral Proficiency Test, the functional 
reading Literacy Test> and the Decoding Test)^co^,d be analyzed to study the 
extent to which language prpficiency was a multi-factoV skill* 

' To recapitulate, the. main ideas in this section: the , principal measure of 
the effects of instruction is the scores on the Oral .Prof iciency Test* We 
asked the question: to what. extent do teacl:\,ers who use *dif f er^nt methods 
teaching differ in the degree of' oral proficiency their students have acquired 
as measured by this test? We also asked: to what extent has the instruction 
had other effects such as the acquisition of simple reading and decoding 
skills? We also asked if two other measures of knowledge and proficiency in 
ihe English language (John and Morano Tests). were related/Jl^^hese measures 
(Oral PrDf iciency. Decoding and Literacy Tests). 



.•EHE MEASUREMENT OF TEACHING PERFORMANCE 



in 'this study we were concerned with Kow teachers teach English as a second , 
language. The wotds, "how they teach", imply that we wanted. an accurate descrip-- 
vtion of hov^ the teachers organized the classes for instruction, what' taaterials they 
used, how. they int^acted with^a[i|- students, and the content that they ^taught. The 
best way to compile a description of this kind is to observe what the teachers d9 
as they, teach cEtss. We therefore assWed obsetaz^&rs to make daily observations 
of each of the teachers in the classes •of the AduU* Learning Center. 

The method 'o.f observation (described in a- later cKapJfer) was . developed by 
observing the teachers for a' period of several months.^ The purpose of these pre- ^ 
liminary observations was to familiarize ourselves with how the teachers taught. 



A category system which described the teaching activities jwas constructed from thi 
information,^ this category ?ystem was then tried gut systematically and further 

• refined. The finaj. product was a set of categories.- and a method for observing 
that provided descriptions af the activities typically occurring in the classrooms 

• (unusual or infrequent activities were noted by the observer when they occurred/. 
During the accrual observation the observer checkeH continuously those categories 

• which' described what the teachers and students were doing, 

V ' ' Theoretical preconceptions did. not determine what we should or would 
observe in a ^ilass* The categories, however, do include descriptions of teaching 
performances associated with two different theories of language instruction, .but 
the reason that the system includes these categories is 'that behavior relevant to 
them bad been observed in the classes of the, Center. These two methods are the 
audiolingual and the "silent way". The b^sic eleme?its of the audiolingual method 
require the^eacher to' model appropriate speech, elicit students' responses, and 
give correc1d.ve feedback. Different teachers" combine these Elements in different 
ways; for example, some '*|^achers do relatively little modeling whereas others 
follow regularly a sequence 'of model, student p^^tice, and corrective feedback. 

The "silent;^ way", method relies heavily on non-A>erbal cues from the teacher to 

*elicit speaking" responses on the part of students. 

Since these two method^ are used by different teachers in the Center, the 

' ^ ^ . . . ^ 

category system reflects^ what we observed them doings and contains categories to 
fully describe either method. It is possible, therefor^, to study three problems; 
(1) do the teachers adhere to a method (such as tfe^e audiolingual): or vary among 
themselves in h9w they use it; (2) is there any evidence that one method is more 

effective than another, either .for all^ students or certain kinds of students; 

* * . ' ' 4^ 

(3.)|are there elements in either method which*^^ particularly effective? 

- ^ • ^ . -I • ' . * 

These two methods are anchoring frames tor teaching styles. The methods 

are not :used 'mechanically or in a ro^e fashion. One»*of tjie goals of this 



research was to study the actual teaching styles, used by the teachers in the 
Ceater to see how either of these two methods was u^ed in practice and how 
elen&nts of both may have been used by a particular teacher. - . . 

The observational data were gathered aftef the first testing;^of the 'students 
and vas terminated shortly before the second testing. The logic of this methodl 
is that changes in student performance from. the first 'testing to the second 
testing should be related to what the teachers did In the intervening period* 

^ - . ANALYSIS OF THE DATA ' / 

Because the data are complex ^nd' particularly^ rich, they may be analyzed 
in a variety of ways. An important problem to be solved was to' reduce the 
number of categories of the observational data. Several different analytic 
procedures* were used to uncover the dimensionality of these data.^ The day- 
school data'could^be reduced to nine factors, and the^^^R^ht- school data to 
eight* To make the analyses for both groups comparable, nine faetorswere used 
in all Analyses of the observational data/ The factors were readily int^)<fi^etable 
and carrespond to our judgments of what yas occurring in the classes. 

• The student achievement data were analyzed in two ways. First,. the usual 
paychlmetric analyses were performed such ar~item analyses and interval 
consistency reliability estinlation. Second, scores^^on all the tests were 
intercorrelated with' each other and with measures of student characteristics. 
- Factor analyses we"!^ also performed on these 'data. . The ^Jirpose of these 

ffc'tor analyses was to find out if there was an underlying structure in the 

' . ' ' - - " * - ' ' 

data.; for example, we wanted to know if the measures of proficiency in English , 

were, measuring the same^r different dimensions. / ^ 



The major data-analytic aim of the study was to relate the data on 
.teaching performance to the stu^ient^gichievement data. The measures of achieve- 
ment focused upon proficiency ift speaking English, decoding skill and functional 
reading skill. Regression methods were uged to relate achievement to classroom 
interaction experiences. In all analyses th^ students' initial level achieve- 
ment and background characteristic^ were taken Into account. Only those students' 
data were used wh(^ were tested ^at tRe beginning and end of the study period. 

, The ''following chapters present each of these analyses. The second chapter 
•* 

contains the description of t*he^ program, information on the test§ used — ^how an^ 
why they were constructed, the students' performances on them, their reliabilities 
and intercorrelations , and related informatipn-ras well as'stflident and teacher 
background information. The third chapter descrl-bes how the observational system was 
constructed and. used and the analyses of these data. - The fourth chapter presents 
tlie results of relating student achievement to teaching perforitiances. 

We conclude^ in the fifth chapter with a stal:emerit about what appear to 
be* e^f^tiya^teaching perfoirmances and practices which should be regarded as; , 
>ypotheses. this study, because of its e?cploratory correlational design, could 
not confintf in"' the strict scientific sense that any teaching performance directly 
affected achievement. We could infer, however, that it was likely that it did 

because^ of our statistical contrbl for students' background characteristics and 

- . ^. ' . ^ r (' 

initial level .of achievement. Replications of this study with other teachers or 

> " ' * . • • * ^ ^. ^ 

systematic experimentation is needed before definitive prescriptions about 

r 

effective teaching performances *can be made. When drawing concltisions about 
effective teaching practices it should -be remembered that thereLoiay be other ^ , 



practices which may. be as effective but which were not observed being Aised 
by the teachetfe in this study* If these* qilalificatd^ons are kept in mind, 
the results of- this study can be used to improve our understanding of the 
teaching of English as a .second language • ' 



• * ^ CONCLUSION • ' / 

• . • • ' * * • 

* * * * ■» 

This 'chapter, described the purposes and methods of thB study conducted on 

teaching perfonnance and student leamin^g at the West New York Adult Learning , 

Center^ The following chapters will describe ,in detail the methods used and 

the results of th^ study. • The methodology of the study provided information . 

about the acquisition of speaking, proficiency in English and about how English 

as a;second language was taught to adult learners. The ^oal of the study was 

to determine what relations existed between how teachers ^ught- and how-much 

students learned. , * * , * 
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PESCRIPTION OF THE PROGRAM* 
Day-School Classes 



Fourteen day school, classes taught by six different teachers were 
available for atudy. (A 15th class had to be .dropped from the study inl 
February when the teacher for another job,) The majority of the cXcisses 

met for an hour-and-a-half a day* five days a week; however, two classe's 
met Only one hour a day, one ii^termediate class met for an hour-and-a-half 
but only three times a week, and an tidvanced class met for an hour-and-a- 
half twice a week. These differences were taken into ^account^.iji' the analysis' 

All classes met at the Adult J^earning Center which is located in ^ 
an office building in dbwntown West New York on the main bus line, / 
although many of the students lived within walking distance. The "Center 
operated on the West New York school system calendar and-all school ijacations 
holidays and snow days were observed^ 

Night-ScTiool Classes 



A sample of six teachers from the 22 available was, chosen according 
to the following criteria: (1) they did not also teach in the d^^hool; 
(2) the^tr classrooms had' a sufficient number of "representative students** 
as defined by. the student background informat;Lop; -(3.) .fhere was a range among 
classes in student competence leve'i and (4) there was a range among teachers 
in previous training and experience. Five of the six passes met for a two- 
"hour session three evenings a week. The sixth clags met only two 'evenings 

a week. * < ' , ' < 

.All classes were held in Memorial High School which is convenientyto 
public transportation, although again many students lived within walking 
distance and many otKers drove to. class. Night-school classes were als.o 
held according .to the public school Malendar. ^ . . " 

■ . ■ ■ ■ • " ' 33". ■ ' 



STUDENT MEASURES: 'BACKGROUND INFORMATION 



In addition to the battery of tests (discussed below)'^whi^ch were administered 

i, 

to assess student learning, each student supplied background information whiqj^ 
p.rovided a concrete description of the total sample' and allowecl us to ^est fot 
'comparability amongisubsets of the students and to ^investigate ^he relation 



'between- certain"^ background variables and present learning^^^' 

Student Information Sheet . 
The b^ckgir^und inf-ormation collected on the Student Infoinnat^-on Sheet was as 

% 

follows: 

1. Sex r ^ . ^ . * ^ ^ 

♦ • 

2. High school; diploma ^ ' ^ 

3. Age - 

4. Education level ,,,,10^00^^^^ ' - .? 

5. Time in United States / ' . ' ' 

6. Study of English in former country ' . , 
- 7. Study of English in UniteH States _ , 

8. -Country of origin ' , * 

. 9. Occupation in former country 

V* ■ . # " ^ 

3.0. • Ocicupation in United "States, . • ^ 

\.«This- information was coded (see Appendix A), keypunched and adde4 to the 
%^students' file^on the computer.. \ . . ' . 



QescrfLptive Statistics . * / 

Day . School ^ ^ »• ^ , , , ' . 

Background Information was collected for 148 day-school students^ Table 
2.1 presents the descriptive statistics f or ^the total group. and by level for 
the •day-school preies^^^iple and the matched sample.^ The matched sample contains 



TABLE 2.1 

. Day School Stu^^nt Background Information — 
p.retest and Matched Samples: Descriptive Stat:isM<:s 



Pretest 



Matched • 



Nuaber 

Sex: M 
F 



^Iploma; Yes 
No 

Age: Mean 
Hange 

Education; 
Mean Years 
Range' 

Time in U« S. 
Mean Years 
Range 

Former English 
Mean* Years 
Range 

English in U. S 
Mean Years 
Range 

Fortner Country 

Columbia 
^Cuba 

Dominican R* 
Ecuador y 

Fen^: ^\ 

Other 



Total - 
Freq> Percent 

148 



Freq, by Level' 

1 2 . 3 



54 
94 

63 
83 



36 
63 

56 



52 

22 
30 

17 
.84 



42 

.21 

2r 

15 
27 



'54 

11 
43 

31 
22 



46 44 44 
21-69' 19-70 24-73 




Tocfcl*^ ' 
greq> Percent 

81* 

26 
55 

32 
49 

46 
19-70 



10 



Freq, 



by Laval 

2 , 



1«08 
0-12 



.58 .83 '1.83 
0-5 0-6 ' 0-12 



6 

,1-24 



1,17 
0-12 





7L ' \ 


28 


29 


32 


9 


13 


, 4 


68 


15 


-15 


25 ^^ 


%. 








40' 


, 8 


10 


, 14 


60 ^ 


li 


18 


15 




48 


45 


45 




21-69 


19-70 


. 24-67 




* 9 


. 10 


11 




4-16 


'4-16' 


4-16 












6 


6 


7 




1-15 


1-17 


. 1-24 




» 








1*83 




0-5 




0-12 




,50 1.41 1.3? 
0-1.5 0-12.0 0-3.0 



'4 


4 , 


0 


42'^ 


35 


^6 


2 


b 


1 


1 


0 


1 


.3 


0 


- 1 


'0 


3 


, 5 



1.17 




0-3.0 










4 


73 . 


90 


I 






I 

. 0 


'3 





-1.25 1.50 
0-2.5 0-3.0 




Occupation ^. 

1. Foranan 

2. Craftsman 

3. Semi-s|J.lled Worker 

4. laborer 

S» Household Worker 

6. Personal Service 

7. Fir^man/Policeraan^ 

8. Ptofessional 
"9. Technician: 

10. Farmer , , . 

11. Farm Worker 
.12. Business Owner 

13. 'fianager/Officiai 

iy>^- Saigi^rson 
16 ••Housewife' 
17. . Unemployed 
4.8 ♦ - Student 




Hatched 



2 


, 3, 




016 


0/0 ^ 


0/0 


2/7 


0/9 


0/0 


0/0 


. 0/0 


0/0 


2/5 


0/0 




0/0 


8/0 


10/1 • 


I/O 


3/1 


e/a 


0/0 


0/0 , 


0/0 


0/0 


0/0 ^ 


1/0 


• 0/1 


efi^ 


6/3 J 


.1/3 


*2/3 ' 


h/2 


6/10 ^ 


0/6. 


0/1 * 


-3/0 


2/0 • 



Pravious'^country/USA . 

See Appendix A for definitions of categories. ^-^j 



those students for whom we had complete test data from both pre-^and posttest 
administrations. Class level was designated by^ Center personnel: levfel 1 = 
beginner, leyel 2 = intermediate, Igvel- 3 ?= advanced. (Students are assigned 
to classes On the basis oi their John Te^t scores'.*) As is evident from the * 



sizes of the^e two samples (148 vs, 81); there^was a 4>,percent j^ttritiou rate 



between November and June for day-school studept'Ss, ^ ^ . ' f 



While examination of the descriptive statistics in Table ^^.1 indicated 
that the two samples were similar, this, assumption was* tested ^statistically, 
^nventional t-tests were run. On each variable to detect any significant 
differences bctweert the students that dropped out <N = 67) and those that ^ 
remained (N = 81). ' None of, the differences were significant. While there 
was a trend for the students who were ]Jouriger, housewives, semi-stdLlled workers 
or' xaneraployed to drop out,- the proportion^ within categories remained similar 
in*the^pretest and matclaed samples.' . , " _ - 

We can thus^state with confidence that in spite of tl\e hi gh^^ attrition 
rate and the resulting smaller sample .on which tfo do our analyses, the day-school 
matched sample is representative of the type of students who attend classes at 
the Center. . , . *• * ^ . c 

Night SchooJ. ' . . . ^' . ' . 

. ' Backgrpund information was cpl^lected for 181 night-'school students. . 
.Analysis- of these demographic' data suppliW the criteria for selecting* , 
a sample of night-school students. This procedure assured that the s'ample - 
chosenilwould be xepresentaeiVe of i:he .types of students attending night- 
school classes. 



* ' 



^Since- complete test data were requireji for our analyse^, this figure mayf be 
slightly inflated. . • / , -• 



Table 2.2 presents the descriRtive statistics oil the background variables 
for the night-school sample. Since^ there "was*" so little attrition betweeh the 
pretest (N - 46) and matched (N = Jl|^samples, otily information for the matched 
sample is- given. « ' ' 

' s . 

Comparison of Day and Night-School Samples 

An examination of Tables 2.1 and 2,2 reveals a number of differences 
between the day and night-school samples. Jhe percentage bf males is higher 
for the night-school, (A4 percent vs. ^^36 percent), fewer;,students have their 
high school diploma^^ (17 f)ercent'vs. 43 percent), the ave^dg^a^e is younger 
(42 vs. 45). In addition, ni]ght-school students on the^verage Have studied 
English^in the United States six months lohger (1.67 years vs. 1.17 years) 
than day-school students, and only 61 percent come from Cuba aa compared to 
90 percent 'of the day-school students. Also, predictably more housewives 
and the linemplciyed attend* daytime classes. * ^ ^ 



STUDENT MEASURES: TESTS 

Three different measures were used to assess students' learning: a set 
o£ Literacy Items,' .an Aural Decoding Test and an Oral -Prof iciencjr Test. . In , 
addition, student scores for two measures regularly given by the Center,, the 

k • • « ' ' ' ' 

Jpb6 -^nd Morano tests,, were entered into 'the data base. The development and 
administration of each measure is discussed separately below. 



Literacy Items 

Tfie s.et.oi 50 Literacy Items measures listening and reading comprehension. 
The set was adapted from f70 items originally developed under the Right-to-Read 



TABLE 2.2 * * ' 

Night School Student Backgroimd Information ~ ^ 
'Matched Sample: Descriptive Statistics' ^* 



Frequency Percent ♦ 



Frequency By Level 



Number 



H 
F 



Diploma: Yfes 
No 

Age: Mean 
Range 

Education: Mean Years 
* EUnge 

Time In U. $. 

' Mean Years 
Range 

Former English 
• j Mean Years 

.Range 

English in U. ^ 

I Mean Ifcara 
j Range 

Former Country ^ ^ 

Argentina ^ 
Columbia^ 
Cuba 
Ecuador 
" Guatemala 

Other /I 



' 41 

- 18- 
23 

7 

42 
17-64 

10 
3-18 



5.9 

lrl5 



1.17 
0-8 



U67 
0-5 



2 
3 
25 
-4 
2 



44 
56 

17 
83 









21 


lA. 


& 


8 


9 


1 


13 


. 5 


5 


3 


2 


2 


18 


12 


4 

t 


44 


42 ■ 


37 


17-64 


22-56 


30-45 






7 0 > 


li 


12 


3-18 


6-16 


8-16 


X 






5;5 


5.3 


8.8' 


1-15 

f 


1-13 , 


4-],3 


llOO 


1.08* 


2.0 


0-7 


0-8' 


0-^5 


T.17 


. 2.41 




0-3 


0-5 


> 0-4« 



5 




0^ 


1 


1 


7 




0 


• ? . 


1 


61 




• 15 . 


7 




10 




3 


.1 


0 


5* 




: 0 


2 


0/ 


12 




3 


1 


1 




Total 




2 • 


2 




0/0 


;'o/o 


0/0 


.0/0 




4/7 


3/4 


1/2 ^ 


0/1 - ^ 




4/24 . 


1/15 


3/8 


.0/1 




1/2 


0/1 


0/1 


- 1/0.. 




. 0/9 


O/O 


0/0 


0/0 




3/2 


2/1. 


0/0 


1/1^ 




0/0 


' 0/0^^ . 


0/0 


0/0 ^ 




6/0 


1/0 V' 


3/0 


-2/0 




. 1/0 


1/0 . 


0/0 


0/0 




0/0 


0/0 


0/0 


0/0 




o/p.* 


c.0/0 


0/0 


0/0 




0/0 


0/0 


0/0 


0/0 




1/1 


- 0/0 


1/1 - 


0/0 - 




1/0 


1/0 


•0/0 


0/0 




•7/1 ' 


6/0 


1/1 


0/0 




1/0 


'1/0 


.0/0 


0/0 ' 




.3/0 


2/0 


0/0 


. 1/0 



"Occupation 

i. Foreman 

2.. Craftsman , 

3. Seai-skllled Wbrker 

4. Laborer • 

'5. Household Worlcer 

6. Personal Service 

7. Fireman/Policeman 
^8. Professional 

9» Technician 

10* Farber 

lU FaA Wbrker. / 

12. ' Business Owner « 

13. Manager/Oi[flcial 
14* Salesperson « ' , 
16* H9U8evife 

17*, Unemployed 

*.18. . Student 



Prevloiifl country /USA , 
lfit% «M>pendlx A for category defixiltions* 
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project.^ All 170 items were ranked in order of difficulty and then reviewed ^ 

for appropriateness for this population. Items deemed appropriate were then ' -^^ 

pilo't; tested with an ESL student with little or no English proficiency and 

an advanced student with proficiency. Th$ test was then revised to. include *» ^ 

more items at both extremes, and at the same time shortened to b total of 

50 items. Parallel instructions in both Spanish and English wlr'6 written. • 

•4 ■ • y ... 

Instructions on how to indicate the answers were given -in either language 
(depending jon student level of proficiency) to insure that students understood 
how to do the taski The ques-tion for eaqh item v^s 'read to^ all students in • . 
English; tj?ey then re'ai the item and indicated the answer to the„ question by 
circling the appropriate word or sentence. Testing was stopped when the student 
answered five consecutive Items incorrectly. A copy of the Literacy. Items can 
l?e found in Appendix- A. ' . . ■ 

Aural Pccod£ne Test • . •«e 

This tesr measures decoding "skills . 'Part T ^s concerned ^th recognition 

of syilables'"and ropt words while Part li.te^ts for phoneme-grapheme correspondency 

* ■ ' • ■ • ■ ' . *3 • 

•f\ie test was originally developed for the Beginning Teacher Evaluation Study. ^ 

Tests at two difficulty ^levels were pilot. tested with .the above mentioned _ 

ESL students and" the results indicat4d that the more difficult test should^ be 

-used. Instructions were modified such that the item stem was read to tlie student, 
rather than the student reading it to himself. Again parallpl instiructlona were 
prepared in both English and Spanish; however, test items We re^d only ii/ , 

- English. A copy -of the Decoding TeS-t can be found in Appendix A., 



?Murphy, R. T. Adult Functional Reading Study. Educational Testing ^Service. 
Princeton, New Jersey, 197-3/ " _^ . ^ 

^Conductet* for- the California Coiranission fox Teacher Treparat ion and Licensing 
by-ETS. FrederfcW. McDonald, Project Director. '• . ^ • 



•AdmlnistraCtou of Literacy and Auiral Decoding Measures 



M ' , & 



ThV tester-observers were trained\how to administer the Literacy Items and % 

fte Auirai^ Decoding. Teit and practiced giving them to several people at the 

different levels who were drawn from^ tbe^ Learning Center ' s^wai ting list, 

' • » * . • 

The te^ts were- adpiiniste^ed to small^groups of students in ah unused class- 

room by the tester-observeirfev. The bilingual tester-observer tested, all the 

- ■ ■ \ . ■ . 

beginning students so that the -directions could be g.iven in Spanish ^nd stuaents - * 

would therefore nbt be penalized for lack of comprehension of spokeiwEnglish as 

Tar as knowing how to indicate tfieir, answer. ' ^ ^ m ■ 

All daytime students were pretested at the Center^between November 5th 

^ and 25th, 1975 and posttestecl^ietween Aparil 5th and May 18, 1976 on thes^ two 

measures. Night-school students ,werfe pretested on both measures between. Aptil 

5t;h and 29th, 1976 and posttested on tihg Literacy Item^ between JOne ast and 

11 th, 1976,-^^^,^^ ^ * • . . ^ 

Oral Proficiency Testi , * ' ^ 

It was originally planned to give an oral proficiency test as a pretest;^/ 
Once the project began in Oct^er we searched the ETS Test Collection, for ^ 
available teslte. We found none which led us to Believe that such tests as ^ 
hdve been reported to exist are . available only . in "fugitive" documents. We ' 
nexl discussed with P. Woddfordf^f ETS the systems he had us^d ii;i., evaluating the 



>^ language-Speaking, qualifications pf ^P^ce^ Corps Vpliinteers. We 'found that 

these systems involved an interview between a yoluateer and an eiroert in the 



language. This expert followed an unspecified str^uctutVfrpm ea6y language 

re^nses to the most difficult, but adapted the interview as he or she.proceede^* 

* ^ . . . * - J V i/*t : X • . ' • ' 

^ The evaluation of the^ volunteer was ^pl^y^ the product 'p!f:-the expert Judgment 

of tte interviewer. Such a system while isatisfactory fiCF? cthe purposes of the 



^ 



-31- 



Peace Corps was not satisfactory fo^lthe purposes of this stud^. For this 

study It* is necessary 'to have an instrtiment that can detect a range, of • ^ 

♦ • . ' . . <* 

differences iij speaking ability which in turn means that the instrument must 
be similarly Structured for evety person to b^ interviewed. ' 

. We next reviewed the work x>f. Politzer of Stanford, an expert in 
bilingual educatioii and the teaching pf English as a foreign language* 
Politzer had developed an oral proficiency test,, but.it had been used with 
elementary school children au4, its content was obviously inappropriate for 



use. with ,adults«. 



^^We also* reviewed two tests developed by^ specialists in teaching English 
as a second langaug6. Psychometrfc data were.noti available for these tests, 

^ such \as .their reliability. But a more important 'consideration led to the 
decision that these; teWts would not be appropriatjs. Their content did not 
match the objective's of the^Leamfng Cetiter preci^iy enough. 

It was apparent^ at this point that it wduld be neces&ary to build a 

\^ -test specif^aily for this projects lyb problems had to .be solved. First, 
we needed to obtain a comple.te and concrete description of the objectives of 
eacK of the teachers in the^ Center. Second, we needed to see what similarities 
there \^re among these objectives, and how the teachers differed in te;^s of 

• . ... •' •■ ■ ' ; • 

the objectives, they were trying to achieve. 

A preliminary set of items was developed by KatKleen Durnin, Meredith 
Stone and Patricia Elias b^ed on lists of objectives supplied by the Learning , 
Center teacher^s./ Each teacher provided us with a list of 20 objectives. After • 
a review of these statements of objectives it was obvious that we would, need 
-to' build criterfion-referenced scales, related to the objectives* and find a 
way of creating a test from whicii the 'performance .of students could b^e tracked 
with' respeqt to the objectives. ^ \ . ' ' \ ' ^ 



The. teachers' lists were combined and overlapping objectives eliminated 
producing a list of 132 objectives across seven teachers I Each teacher thea 
rated the 132 objectives^ on the following: (1) whether or not it was something 
they taught; (2) how important they felt It was for proficiency; (3) how much^ . 
emphasi3 they gave it in their class; and (4) how difficult they thought it. was 
to learn. The Proficiency Test would be constructed using those objectives 
on which there was high agreeiaent among teachers teaching at the same level. 

The work described above was carried on from October to December. By 
December it was apparent that the oral proficiency test could not be built 
to the specifications agreed upon in time to use it as a pretest. We d^cid^d, 
therefor^, to use the Oral Proficiency Test as a posttest and to include* 
analyses which wpul4 relate other indices of initial proficiency (John and 
Mbrano Tests, student background information) to the Oral Proficiency scores 
obtained later. • *" '\ * 

During February and March thd project director and associate project 

director develoged .several series of items based on representative examples 

irom the teachers* objectives for, each level. These items were in a structured 

conversational format an^ Arranged in order of difficulty by objective (based 

on semantic and syntactic complexity). In* addition, questions concerning two, 
^ . , * . * , . • . ♦* 

magazine "action" pictures*' and translation of three Spanish cartoons were used 

to elicit English responses. The. first two components of the test (structured 

•conversation and action pictures)* were designed to elicit increasingly complex 

forms of the verb phrases in terms of tense and prepositional and 'adverbial 

qualifiera. The third component (cartoons) waa designed to assess the students 

ability to go froii\ idiomatic Spanish to idiomatic- English. Items w^re reviewed 



by ' t he > coordinating teacher and then pilot tested by the coordinating teacher 

and associate project director with srtidents on the waiting list. Items were 

ti ^ 

modified or deleted until a 60 item test requiring approximatley 20 minutes 
to administer emerged^ 

The purpose' of the Ofal Proficiency Test was twofold: to assess botlj^ 
conrprehensio'rt' and the ability to generate appropriate English sentences. 
Therefore, the test was scored for comprehension, use of appropriate structure, 
correctness and number ^f prompts for each of the items. Again, instructions ■ 
were given in either Spanish or English, although test items were given only 
in English. A copy of the Oral Proficiency Test and its score sheet appear 
in Appendix A. v ' • 

Administration of the Oral Proficiency Measure, 

' The coordinating teacher and the associate project director ^trained the 
tester-observers to administer the test. Students from the Center who were not 
'part of the saftple were subject's for .practice test sessions. Each test was 
administered individually in a separate room and tape recorded to allow fbr^ 
review or verification of scoring. Daytime students were tested between 
May 26th and June 11th, 1976. As additional^ testers wiare needed to complete^ 
the night-school testing, two other teachers were^lso trained and practiced 
giving the test. Night-school students , were -tested. between June 1st and 
June 10th, 1976. Again beginning students were assigned to bilingual testers 
so that directions could be given in Spanish'.^ 

John. Test , , . 

This oral proficiency test was -developed by Linda Kunz at Hunter College. 
It consists of eight pictures about which the student is asked 22 questions. 



In addition to the ^comprehension score the student. is 'also r^ted on fluency, 
Structure, pronunciation and vocabulary by the tester.* A -copy of the test 
and its score sheet are in Appendix^A.x * \ . - 

/ ^ * ■ 

^ This test is regularly given to each student when applying to the Center"* • 

and again at the end of each year^ Student scores are then used for placement. 
Thus scores from this test inay have been obtained from day--students anywhere 
between April, 1975 and September, 197^^^Jpw»«^, ' the night-school students 

^ were all teated (or retested) between March 18th Vnd April 12th, 1976. Dfespite 
this, variation in testing dat4s and the fact that ti^ny, if not most, of the " 
students ^had been given the test at least once before, scores foi; the John 
Test were 'included in the analyses because they were found to be highly 
related to other indices ^of initial status. 

' ^ Morano Test 

»/ * . ■ 

This test is a paper and .pencil test of recognition of correct use of 

English ^gr amm ar. It has 50 items each, consisting of thiTee sentences expressing 

the same idea, only one of whfeh is grammatically correct. The studett is 

instructed to read the items* and indicate the correct sentence. A' copy of 

the test is in Appendix A. . . * 

THis' test is also regularly given by 'the Center .to' its applicants. 

Although the scores are considered during placentent, they are not given 

anywhere near the same height as the John Test scores;. The day students* 

scores were obtained between April, 1975^ and September, 1975, while all riight- 

students were given the test between March 18th and April 12tja, 1976. The 

Morano Test scpres were included in the ^nalys^s because they were related 

to other indices of initial status "and allowed us to control for differences 

in proficiency prior to the period of this study. ^ 



Descriptive Statistics 

Table ^.3 presents information cpnoerning each' of the aforementioned 
tests. The' group, (day or night-schopl sample), number of students, and 
bime coliimn -headings are self^-^cpl^natory.^ The mean (X) is the aV€?rage ^ 
score of the^^udents taking the test. The range gives the lowest and\ 
highest:i,*scores~u>b-tained by the group of students. The stan^da.rd , deviation , 
(SB) Is measure of ^he' variation in scores with^in the group of students. 
The. reliability coefficient is a measure of the internal consistency of an 
individual's responses to all the items on the -test. You will note that the 
reliabilities are consistently high, with the one exception of Aural Decoding II 
which is in the "mid 70's. . * ^* , - . 

Due to the possibility of selective attrition in the day-school sample, 
t-tests were computed to detect significant differences in pretest scores 
.between those students who remained in the sample (N = 81) and tl^o^^|!^o 
dropped out (N = 34-67 depending on the test). No significant^ differences 
were found . There^'was a slight trend^for those who initially scored higher 
on the John and Morano^test^ to drop out; however this trend was not found 
for the Literacy or Aural Decoding Mea'sures. ,^ , . . 

Table 2.4 presents a comparison of scores for each test, pre-.'and_ post, 
between all students taking the test (Total) and the Matched samples for 
both day and night-school students. An examination of the mean, range, and 
standard delation fi'gures listed indicates that in the cases where there 
are differences, these differences never amount to more than one. or two 
points. This result confirms that our day and night-schcfol matched samples are 
representative of theirvrespective larger groups. Thus all further discussion " 
of analyses will concern only data for the matched samples.^^^j. 



TABLE "2. 3 



STVDENT ,TESTS: .DESCRIPTIVE STATISTICS 



Who . 


When 


N • 


■ I 


Range/ 
\ 


■ 1° 


Bay 

voy 


November (Pr,e) 


148. 

1 '1 Q 


29.82 • 

'57 Q1 


3-49 

11 /• Q 


12.52 

7 00 
/« ZZ 


Night 
Night 


March fPre"^ 
May ^(Post) 


45 
43 


34. 73 
36.81 


4-49 

ii-4a 


9. 70 
9.98 


Day 
Day 


iJovember (Pre) 
. April (Post) 


148 
120 


26.72 
28.42 


6-38 
13-37- 


7.40 
5.33 


Night^ ^ 


March (Pre) 


45 


29i47 


14-37 


6.90 


(> 

Day 
Day 


'November (Pre) 
April (Post) 


148 
120 


47.84 
52.45 


31-57 
36-^ . 


.'-5.28 
4.63 


Night 


March (Pre) 


45 


^.33 


47-59 


4.86 


Day ' 
Day , 


November (Pre) 
Apfil (Post) 


148 
120 

45 


.74. 57 
80.89 


49-93 
56-95 


11.03 
7.90 


Night 


March, (Pre) 


78. 80 


60-96 


9.47 


Day 

Night ' 


.June (Post). 
. .June (Post) 


113 
43 


30.48 
27.65 


2-58 
3'-53 


14.66 
16.21 


Day • 
Night 


June (Post) 
June (Post) 


113 
43 


11.83 
10.49- 


•0-45 
0-29 


'8.84- 
8.65 


Day 
Night 


June (Post) " 
June ' (Post) . ' 


113' 
43' 


14.53 
12.09 


0-41 
.0-33 


■ 10.16 
10.01 


Day 
Night 


Juijie (Post) • 
June (Post) 


113 . 
43 


9.31 
' 7.70 


.*0-^6- 
.Orl9 


• 4.86 
4.47 


Day* 
Nigfit 


November. (Pre) 
March (Pre) 


115 . 
46 


37^7 
30.54 


■ 0-70 
0-65 


20.54 
21.07 


Day * 
Night 


Noveipber (Pre) 
' March , (Pre) 


118 . 
46 


26.82 
26.3b 


. . 3-50 
7-46 


' 11.47' 
10.46 
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Comparison 
Matched (MAT)' 



of Test Statistics for Total (T0.T) and . - 
Samples for Day (D) and Night (N) Schools 



Instrument 


Whb 


Time 


N 


X 


Range;. . 


sd' 


Literacy 


d-tot; 

^D-MAT. , ' 


Pre 
Pre 


148 
81. 


30 ■ 

30 . . 


3749.-^ 
A8.- 


13 

12; ' 




D-TOT 
D-MT ' 


Post 
Post 


119 
81 


38 

37- ; 


. 11-49 
11-48 


7 

8 • 




^-TOT 
N-MAT ' 


Pre 
Pre 


45 


35 

34 ■' 


' 4-49 
4-48 


10 
10 




N-TOT 
N-MAT . 


Post 
Post 




37 \_ 


11-48 
11-48 


10 

' 10 


Aural 
Decoding I 


D-TOr 
D-MAT 

< 


Pre 
Pre 


148 
. 81 - 


27 
26 


6-36° 
6-38 - 

0 ft 


■ «« 

7' 
7 




D-TOT o 
D-l^T 


Post 
Post 


120 
81 


28 
28 


13-37 . 
13-37 


5 • 

• 6 > 




'N-TOT 
N-MAT ' 


Pre 
Pre 


45. 


29 . 
29 


14-37 
14-37 


\7 
7 


Aural 
Decoding II 


D-TOT 
D-MAT ^ 


/ 

Pre 
Pr,e 


• 148 • 
81 


48 
48 


31-57<» 
31-56 • ^ 

♦ 


- " 5 
6' 




^DtjTOT ^ . 
• D^MAT ' 


Post- 
Post 


120 
81 


• 52 
52 


36-59 
36-58 


5. 




N-TOT \ 
fl-MAT 


.Pre 
^"re 


45 

,41 - 


/ 49 
49 


37-59° 
37-59 


5 
•5 



John 



D-TOT 
D-MAT' 

N-TOT 
N-MAT 



Pr6 
Pre 

Pre 
Pre 



115 
81 

46 
41 



t37 
39 

31 
30 



0-70 
0-69 

0-65 
0-65 



20 

21 
■21 



Morano ' 



D-TOT 
D-MAT ' 

N-TOT 




118' 
81 

46 



27 
25 

27 



,• Pi:gficiency 
Correctness 



3-50 
0-43 

7-46 



ERIC 



11 
10 

10 



N-MAT- ' , ■ 


.Pre ^ 


'^41 


25 . 


7-46 


11 


'v 

D-T^T 
D-MAT • 


Post 
Post 


113 

, 81 


12 " > 
13 


0-45 
■ 0-33 


9 


N-TOT 
^-MAT, 


* 

Post 
Post 


43 , ■ 

- 41 

4'^* - 


. 10 ■ 

IQ 


-b-29 
.0-27- • 


9 

• 8 



RELATION OF STUDENT BACKGROUND INFORMATION TO TEST SCORES 

The intercorrelations' of student test scores, both pre-'and post, wij;h 

^ach other and with cbded student backgrdund variables * (see Appendix A) are^ 

presented in Table 2.5 for the day school and Table 2.6 for the flight school. ^ 

^ For the day-school sample with an N = 81 a correlation above .22 is ' 

significant at the •OS letel and a correlation above .28 is significant at the 

'.01 level. Examination of t,he test intercorrelations in -Table 2.5 indicates 

.that all of the tests are significantly correlated \d.th each other with the^ 

exception of the Aural Decoding II posttest. The subscores of' the Proficiency 

• « 

Testj correctness and comprehension, were highly correlated (r = .86) a^s were 

the correctness and use of structure scores (r = .97). ^ 

The relative magnitudes of the correlations suggest that while the John 

and Morano tests tap quite similar skills (r = .78), the Literacy and Decoding" 

Tests appea? to measure* some different competencies. The Proficiency Test 

correctness scpte falls in between witl> correlations of ^8 and .63 with the 

John and Morano Tests, correlations of .51 and .60 with Aural .Decoding X, and 

correlations of .47 and .59 with the Literacy Test. Tl>e Proficiency Test compre 

♦ 

^hensfon siibscore, however, has a* high correlation with the' John Test* (r = .85). 

, _A number of background variables were significantly related to test sco^res. 

h \^ ^ ' ' \ ' 

While, UP relation was found between sex and test scores, a negative relation ^ ^ 

.w:as. found with agei younger students do better on the Decoding, Ll(teracy 

and Proficiency Testg.' Time spent in the U. S* was .not related, while 

having-a diploma was ifelated. only to .the fall Literacy Test score. A higher 

s.tatys code (see Appendix A) for job in former country was related to all test 

♦ . " ■ ■' ' ■ > ■ 

scores except Decoding II, while a high er code for jjob in U. S. was related 

only to Decoding I and Literacy in the spring. ' * . ' . 
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-TABLE 2.5 

Intercorrelatlons of Fall and^Spiririg Tests and 
Stud*ent Background Information 
Day School*^ Matched Sample* (N = ^81) 



Decoding^ - F 
Decoding2 ^ F 
Literacy - F 
John - F- 
Morano - F * 



Decod4.ngj^ - S 

Decoding 2 - S 

Literacy S 

Proficiency 
Correct 

Compreheijision ' 
Structure 



Dj^-F Dg-F L-F J-F M-F'* 



^•50 • - 
.65 .41 



D^-S , D2~S L-S P-CT 



P-CP P-ST 



.56 .29 



.77 



.55 


.25 


.59 


.78 


.4 






.73 


.40 ' 


.48" 


.42 


.45 






.33 


.27 


.22 


.10 


.15 


.41 


















.47. 


.31 


.71 


- .64 


.51 . 


- -^7 


.16 


.60 


,.26 


.59 


.68 


.63 


.51 


••22 


.70 


.3? 


.75 


.85 


.73 


.63 


.21 


.63 


.30 


.63 


.74 


.68 . 


.54 


. .'23- 



.90 



4- h 



Sex 
Age 

■Time in U. S. 
Digloma^ 
Former j;ob 

r 

Job In'U. S. 
C-Origin 
Years. 'Education 
Former Eng. • 
Eng. in .U. S. 

:P <. .05 for £ > 

•r ' ©"1 for ir > 



.07 .10 

-.32 -.15 

-.15 . * .05 

.. 21 . 08 

.25 .17 

.13 .08. 

.05. -.12 

~ .21 . 12 

.16' .,19 

^ .28 ;i3 

.22 . 

.28 --^ 



- .17 
-.30 
.18 
.22 
.;39 
.21 
.02- 
.33 
.25'. 
.41 



.17 
-.13 
.13 
.11 
.33 
.20 
--?07 
- .18 
.26 
.5T 



.02" 

.05 
-.07 

.OS, 

'.33 

.19 . 
-.'l5 

.16 

M 

.43 



, .03 :^.06 -^.02 .12 



.19 



-.27- -.06 -.28, -.18 -.23 



^ -.13 -..08 .00 .05 



,11 


.17 


.09 










.18, 


^22 


.26 V 


.1.6 


. .24 


.00 


.09 


.04 


.23. 


.15 


.23 






r - 


.08 


.13 


.12 


.25 


.03 


.31 



.06 
.18 

:32 
.20 

s^04 

.23 

m 

.14 

.37 



,13 

-.21- •; 

.04 .. 

.it ^ 

.17 " 
'.01 
-.25 

.16,- 

.23 



Country ,of origin was not related tp test 'scores. Years of education ♦ 

was related only to Literacy in the fall^ (although Decoding I - Fall, approaches 

: * ^ * . . 

signif4.cance) , but to all but Decoding i; in the spring. The study of English 

in' formei; country was related only to John and Morano test^ scores, wliile study 

of English* in the U. S. was related to Decoding I,, Literacy and Proficiency 

scores as well, * - - ^ 

For the night-school sample* with an^N - 41, a jporrelatibi? .of .tiO or* 
above is significant at the ,05 level while a correlation ^f ,39 is significant 
alt the .01 level* . ; 

Table 2. 6^ indicates that once agairv all -of the"t;fests are sig^^Jcantly 
c6rrelated*\*ith ^e^xception of Aural -Decoding II. The three subscores of 
jthe Prof ic^^enc^.Tes^are^again^highJ.y ^related (r's = .83/ i87 and ,.98). <As 

^' ' ^ . ^ : ^ 

w^th the day-school, th'e ^oM and MoVano T^ts were highly cortelated = .770. 
However, cotr&latJLons jjetween the ^rano and the 'Literacy Test scores (r = .71 
and .77) wer^ much IvLgher for t\is sample than for the day-schodl sample. 
Correlations of ail^lhelse test^^th the' Troficien&y^ correctness score were 
also high (r ='.73 to;.78) . o\ - . ^ ' 

Only five of the ten background f a&tors .werel;related to any of, thp test 
"scores. Age was. agair^ negativeLy.^elate^ to DecQ^fing* 1 and Literacy Country 
of origin was significantly relat^ tfo the JPTO&ftlency compirehehs^n sub score 
but not to eitHer the correctness or use*o^ strufcpure subscores. ^* 

' « }^ ■ . ' '^ ' , . . ' ' 

Years of education was related to all .tfest scores but' Aural Decoding II. 

The study of English in former country was' relatj-ed to Aural Decoding II and 

' . / , * 4> ' 1^. ~ ' ' - 

Morano .Tests for the pretests, but ngt to any of^tTie' pcrstt^ffrs^^while study 

of English in the U. S. was related only to the Morano pretest and to, two 

Proficiency subscores, comprehension and use of structure. 
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TABLE 2. 6 



Iritercbrrelatlons t)f Winter and Spring Te%t^^nd 
Student Background Information 
^9 . Night School — Matched SaiBple (N = 41) * 







D2-W ■ 




J-W 


M-W 


» 

L-S 


- 

P-CT 


P-CP 


P--ST 


Decodlngj^ - W 
Decoding-' - W 










■ 










Literacy - \7 


■.Ik 


.14 


- 


• 


• 




% 




• 


John W 


.64 


•10..- 


.71 










•• 




Morapo - W 


.67 

w - 


.36 


.71 


.76 












Literacy - S r 


.74 


.26 


.80 


.72 


'■.11 


m 


• 






Proficiency 7 S 
Correct 


• 

•71 ' 


• 

, .21; 


.73 


.78 


.T8 . 


.73 


• 

• 




- 


Comprehension 
Structure 


.73 
.11 


.29 
.24 


.73 
. 73 


.73*^ 
.77 


*'.79' 
.81 


.79^ 

.74 


.83 

, . 98 


. 87 




Se!x . 


" -.11 


.05 


-.06 


.68 


-.09 


-.02 


-.14 


-.09 


-.15 


• .Age V 


-.42 


-.23 


-.31 


--.18 


-.17 


-.41 


-.21 


-.20 


-.17 


Time In U. 


.03 


-.11 


.17 


.27 


.15 


.15 


.22 


.22 


.22 


* Diploma 


.29 


.20 


.12 


.26: 


.22 


.23 


.18 


.24 


.14 


Former Job 


.13" 


- .13 


.00 


.04 


.07 


/.08 


-.16 


.07 


^ -.08 


. Job In U. S. 


- .13 


.12 


.16 


. .19 


.11 


.26 


.25 


.23' 


.23 


tt-Orlgin 


.22 


-.05 




-.00 


• .04 


.26 


" .27 


.31 


.24 


Years Education 


.56". 


.29 


.45 


.51 

* 


■4. 

.48- 


.65 


.4t) 


.61 


.45 


.^^JFormer Jfng. 


.22 


.35 


'42 




...30 


.26 


..11 


.26 


.25 


Eng. in U. S'. * 

♦ 


• .29 ■ 


.09 


.25. 


;i6 


.32 


.21 

\ 


■ 7^5 


.36 


.32 



p < .05 for r > .30 > * . , 

p < \.01 f or r > .39 . 
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)CESSING OF. STUDENT DATA . 



. Fpttowing each administration of the tests, the test booklets or coded 
answer sheets wer^^ent to ETS for processing. These booklets^ or answer 
sheets wer^ processed and data sets were made for use as input to the compyter 

( ■ • • ■ 

These data, were .sorted by the student identification number and were edited 
for errors. All errors were corrected and the corrected data were re-edited 
until no errors were found* 

Scorfes for each test were derived by comparing th^ item responses with 
the 'answer key for the test and counting the number of correct responses. 
Item analysis, preliminary item and Score correlations, :and preliminary 
summary statistics were then generated. 

After the final" test administration for each group of students (day and 

"* » 

night), the prie- an4 post^-dafa were matched. (Day and night data were kept 



on separate^ data sets and^were analyzed separately). More analyses including 
score correlations were made. ^ . 

Student background data and class attendance data were collected. and _ 
sent to, ETS. These data were processed and data sets were made which were 
sorted and' edited' and* used l^s input to various preliminary analyses. 

The studdnt background and attendance data were then me^Mg^fLth the 



matched spring and fall data?Jor more analyses. These data were eventually 
merged- (with the teacher observation data for the final analyses. Please see 
Appendix D fox d^tails^^of the data processing. ^ ^, 



TEACHER BACKGROUND INFORMATIOl^ 

Both day and night school tedche^s fii|ed out a questionnaire, asking 
for the following information: age; sex; undergraduate institution; . 

Undergraduate major; graduate .work institution; graduate work major(s); , 

• * 

nmber of credits completed; degree (s) completed; number of years teaching; 

grade level of teaching experience; number of years Ceaching ESL part-time 

(night-school adults); number of years teaching ESL fulZ-time to children, 
' VT. ; • ' / ' ' . ' 

to teens, to adults; number of ESL seminars; number dr ESL workshops; ^usefulness 
^ ^ • ' ' * \ ^ . ♦.'^ ' 

of undei:graduate experiences; •usefulness of graduate experience; usefulness of 

ESL seminars and workshops; percent of teaching which is audiolingual, silent- 

way, and other; and a statement describing their teaching method^ and philosophy. 

A copy of the Teacher Questionnaire appears in Appendix B. ^ 

Table 2.7 shows the descriptive statistics for these information ^tems 

for. each group of teachers. The ^two groupT differ on a number of dimensions. 

The majority df* the day-school teachers are women, while only half of the*^^ 

night^^Qol teachers are^ All six day-school teacliers have cotapi^ted some 

^graduate work, ^three have completed a Mas^r^s degree; and one is woirking 
on a second degree; while only three of the night-school teachers have done ^ 
graduate work all three" completed a Master's degree and two of them' have 

'completed a second Master's degree/ Night-school teachers on the average 
haye ha'd more experience teaching, but this apparent difference is due 
primarily to one teacher who has taught for 26 years. Most of the day-school 
teachers', experience' with ESL' has bean teaching adults, .while most -of the 
night-school teachers also teach ESL to child.ren of teens during the day as 



well as to adults in the evening. Day-school teachers have attended a few 



TABLE 2.7 



Teacher Backgroupd Information 



Sex 



Undergraduate 
Institution 



^ Undergraduate 
' Majof 



Graduate l^ork 
Institutioi\ 
(1st. MA) 



Graduate Work-^ 
14a j or 



Number of Credits 
Completed ^ * 

1st Master^ s r • 
Degtee Completed 

Graduate Work - 
Institution 
(2nd' MA) 

Graduate Work 
Major 

' ^ - ' /• 
.Ilttoter of Credits 
Completed 
/ ^- 
? 2n<i Master' St Degree 
' Comple|:ed 



' DAY SCHOOL (N 6) 
X: 35 ^Range: 24-45 

5 Females; 1 Male 

Jersey City State: 2 
.'Douglas: 1 

Ladycliff : 1 

Mdntdlair State: 1 
* Seton Hall: 1 ' 

Elementary Ed. :^2 
Spanish/Ed.: 2 
. Philosophy: 1 
Italian: 1 i ' 

'•Jersey City State: .2 
Fairleigh Dickinson: 1' 
Monttlair Stat;^: 1* 
Rutgers*: "1 , * 

Seton Hall:^^^ 1 



Elementary EdT: ^1 ^ 
ESL: j^i -[ [^ ^ 
ESL* and<,^Ed, t J. ^ , 
ESLl' and^ ReadiTig:i"~- Z . 7 
Guidance;>and Tersonne!^ 1 

N =. 6 ; • X : . '-^30^ Range :' 9-4 7 



N =v3 

As 



"Rutgers r; • J 



NIGHT SCHOOL (N = 6) . 

X:^. 35 Range:. 23-55\ 

3 Females; 3 Males 



Jersey ?ity State: 3 
N.E.^ MissouH: /l^ 
Saitit Peterj^s-: 1 . 
St. Elizabeth^s: 1 



pieman ti^yyfid. : 4 
History: 2 



Jersey Gity State: 



\Elementafy Ed. : 1 
"Reading: 2, 



Italian Lang. & Lit.* : 1 
N - IrX: 27 • - , " 



N « 0 



^N.^- 3, X: 32, Range: 30-34 



N*- 3 



Jersey City State: 1 
Montclair State: 1* ^ 
Fairleigh Dickinson: j 

Elementai^r. Ed. : 1' , 
. N =^.3, X* 26, Range 15-3,4^ 



• N = 2 



O .The N for each 'Category 6 unless-otherwise stated* 
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TABLE 2.7 (Continued) 



it - Number of Years * 
• .Teaching ^' 

\T^acbi4ig \ ; 
. Experience Level 

' . Years' ESL . 

_at Night, Adults* 

' Years ESL Children 
\ ; Full Time 

Years ESL Teens 
: 'Full Time 

Years^ ESL * Adujts 
Ful'l Time 



'Number of ESL 
.Seiaiilars 



X: '5. 



DAt SCHOOL 

. Range:. 1-18* 



jjiiimber pf ,^SL 
* ^WorkshotysJ/^- 

ysef Illness \ : 
Undergraduate • 
Experience*** 

Usefulness 'of 

Graduate 

Experience** 

Usefulness of 
\ ESL. Seminars 
and Workshops** 

Percent Au'dio- 
. \jtlngual Method 

Percelit 



Elementary: 3.. . 

kigh School: - 1 

Adults:- 2' . ' 

N = 2, X: 8, .Range: 3-12. 
N = 1, X:' 4 

N = tf • . - 

N -^* 6,, X: , 2, Range 1-4' . ^ 

X:. A, Range: 0-8 

X: 17, Range: 4-30 . 

X:, 4, Range: 2-5 / 

N = 6, X: . 3, Range: 1-4 . . 

. f , • " 

• I ^ 

X: 1, Range: -1-2 

.• - 4 

« , . - ■ », _ ^ 

' \ . . : \ 

~r = 2-, X:- -.62Xv '^gPL ^6Qt*&5| 



NIGHT SCHOOL ' 
X: •iO Range: 1-26 



Elementary:. 4 
Junio.r Hig.h: 2 

N =■ 6,.'X: 4, Range: 1-6' 

N = 3, X:' 4-, Range: 1-7 

, N = 2, X: 4, Range: 3-4 

• * «■ ♦ 

. N =■ 0 . . . . ' 

, 1^ Range: 0-5 
X\ 13, Ran|e: • 2-2^ 
X: 3, Range: ,1-4 ' , ' 

• N = 3,:.X:i 2,. Range: 1-3 

X: 1, Range: 1 



'4 




X: 66^, Range r.^ 30-99:2 

N =»:5, X: 14%, Range: 1-30% 



*The N for each category = 6 unle'ss otherwise stated. 

Very. useful = 1, ntxt at all uaefuj =5. 

' '''Wis;;, • , ■ > 



^ N - 6, X: 60%, 'Ranged, 10r9Q% 
X; 35%, Range: 10-60% ^ 



N « 1,;X: . 30% 
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,>Concepts Underlying 
'^jjethod &'l^hilosopliy: 

Audio lingual 
Cognitive 
Conversational 
" Counseling-learning 
* ^-Ecletic ^ 

Flexible-ad just tp 
*^:^;class 
Group work J 
Individualize 
Listening andk>- 

^speakingf^' 
• Nee3^ f dfe^English 
^ • ^eer te^chii^^.. 

Reading arid 
\ / Writing ' • . 
. Silentj^;TWty- 
Situ^S^nal 
. ^^...sWdent^dominated 
.JSiiiS^t fespon 

sib 
Variety 



•V 




•TABLE 2.7 (Continued) 



DAY SCHOOL 



N = 2 
N = 1 
N = 0 

N 3 

N = 3 
0 

N = 1 

N = 0 
N = 0 

N = 1 
N = 4 
N = 3 
N = 4' 

N ^ '2 
N = 2 




NIGHT, SCHOOL ^ 



* 


X , 


N = 


/. 
^. 






XT — 


1 


N = 


0 




0 • 


= 


2 


N = 


1 


\n = 


0 




1 


N = 


1 .• 


N'= 


2 


N = 


1 , 


N- = 


4 


N = 


3 


N = 


0 


N = 


1 


.ii = 


1 



t 



' V.- 



The N for each category = 6 unless otherwise stated* 



ERIC 



56 
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Tnore SESL semiitars and workshops ♦ kHRx, groups consistently rate their. 
iin;^|^radiiate experience as not very- useful, their graduate experience 
as 'more useful, and th6 ESL seminars and workshops as very useftilt 

While only twa day-school teachers say they,, use the^audiolingual > 
method of. teaching, they estimate thatw^ome 60-65 percent joi^ their teaching 
falls in this category; ' all of the night-scKool teachers st;ate that; they 
use the aUdiolingual method, but their estimates ran^e frop 10-90 percent*^ 
'MX of the teachers i,n both"^ group's state that they use silent-way methods, 
but "the day-school teachers* estimates are. from 30-90 percent while the night- 
school teacliers' estimates fall between 10 and 60 percent. With respect to 
the 17 concept gfvan as underlying their teaching methodology and philosophy, 
only two are substantially different for the two groups: *day-school teachers 
more frequently state their belief that teaching style should be ecletic and 
*that students should doij^inate^ classroom interaction. ' ; 

A discussion of how t\ie Teacher Questionnaire .information relates to ^ • 
* other Analyses appears in ^Chapter Three. - „ . 



\ 
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DESCRIPTION OF raE.- OBSERVATION S?STEM 
IJCvV*,^ \' • - ' * Development of 'the Observation System- •/ ' \ 



5^^^i<^i . DESCRIPTION OF OBSERVATION SYSTEM 



Teachers' methods, materials, and interaction with students were coded ^±th 



axi^ wtn-class oTiseryation system* This system was based on what the ESL teaghers • ' 
5^ „ ^^dy.students .at the Center actually do. T6» develop the system, the^projeQt 

'director, associate project -director and research assistant observed In several 
classrooms and then ^jajj^^^^ their observations vith the coordinating teacher. 

The associate project director and research assistant then took on 
primary ^responsibility for developing tlie eyst^ and spent much of the -f iirst 
three weelc^ of ^the project observing in Center classrooms and talking with the 
tfeachers. The last weak in October. each teacher was videotaped teaching a 
* 30*tonute class.! * , , * , . , 

On* tlie basis of the in-class obser^vations, the^j^^dfifl^apes and discussions 
. wlt'h the project director and coordi-nating teacher, .we developed a preliminary 
form of the observation System. This first system' was tried^out iji the Center 
classrooms by the research assistant. After discussions among the project 
' director, associate project director, research assistant and consultants, repeated 



U-i- 



tnodif icatioii's, .were made ,and a lexicon of definitions prepared* During this time 



"-actual observations V/ere discussed to clarify definitions of categories; 
■ XV ^^7. ^categories .were expanded or 'elifltfnated; videotaped ^and, in-class examples were 
i ' •v considered kn ^ d.eveloping the i fir^al* categories . An in^ortant consideration 
ifci'^ i'^iv^^^^ Wi^W^mic^ W4 thai the Ut4g(ixieL wolild- idescribia — L 

K \ % the differences between .the two major teaching strategies used iiji the^Center. - 
The final draft version was used in^pr61iminary observations of nigHt-school 



classes as well^as day-school classes to insure, its; appropriateness to botli 
settings'* 



' Description of 'the Observation Systeiji 

'The result of the development work described above was a categorical 
, observation system which allowed .for sequential coding of ^lassroom behavior. 
. * A reduced copy of the Observation Coding Sheet is shown in Figure 3*1* • A 
• complete lexicoj:^and a sample observation are in Appendix G. 

Three superordinate categories — context (instructional design) , Tnaterials, 
and strategy (method ^of instijictltoa) — describe the classroom setting within 

which the teacher afid student behaviors are recorded. Efach of these categories 

• ; r .... ' ' 

- is ^subdivided; e^g^ the coijtext can be drill,* writing, explanation, dictation, 

- etc., and each subdivision, has .a numerical code. The^ categories are cod^d 
initially and recorded again only if they change dujring the observation. pelribd.'' 

The first Avgjroup of teacher behavior categories — questions, serial redirects 
, direct ix)n, models, writes oh board, explanatitm, and otker — Are^discr^e 
instructional behaviors which usually initiate a teacher-Student (s) interaction. 
Whe^re these ^behaviors are carried out nonverbally, they are coded with an "N" 
rather than a check mark^ , , The next co^timn — class, gfpup,- individual — signifies 
to whom this behavior is directed. • - ' 

The first group of student beliavior categories — answers, free response, 

^ .practice, writes on board, reads, chooses not 'to respond, asks question, 

• ; " - ^ ; ' ' 

(participates in) conversation, student- to^slhident feedback, and other— rare , 
^ thoae behaviors which either follow .j:he teacher's initial behavior or initiate 
\ ^ X . Interac^tlOn on the part of the student.' Gonyer^tibn jand student feedback 

The next, three categories — positive, corrective, negat^ive — describe the 
possible tyn^s of teacher feedback. Here again, since,, nonverbal feedback is 
an lifiportanF component of ."client w?y** instructioa, an "N'Ms used to indicate 
a nonverbal response. ' . ' * ' *. . 



The.. second group Qf teacher behavior categories — ^models, prompts; asks^^ 
to repeat, repeats^ explanation, writes on board, direction, qjiestion, and 
Other— designate response behaviors. on the part of the teacher. Again, 
tjiey are coded to^.ndicate whether theV are given verbally or nonverBally* 

The second group of student behavior categories — student models, student 
' prompts (these two usually follow a direction from the teacher), answers, 
. free response, practice, writes on board, reads, chooses not to respond, 
r^asks question, (participates in) conversation, stude'Jlt-to-student feedback 

and other—describes those student behaviors given in response to the teacher^' S 
^IrespMse to the student ^s itiitial behavior or response. Again, an "S" is use^ 
to ^We those responses which, were given in Spanish. 

» The comment column allows the observer to indicate what the "other" 
behavior ,co4edr,Qn that linfe. is of. to note^.some unusual' classroom occu^repce.^ 

^ ' ' Use of the Observation System- > ' 

the observation system was designed for in-class use. men the observer 
first entered the room, he or she took a few minutes, to fill out the top 'of 
the observation sheet, indicating the date, teacher, time, number of 
students, observer, class set-up (diagram) and language structure being 
' taught. During this^ orientation time it was possible to code the three' super* 
'ordinate categories and then quickly proceed to 'accurate coding of *i:he classroom 
interact iofn behavior. " ' , 

.By way. of illustration, a possible classroom sequence and its codes 

"T) — ' y ' J ^ i . ^ ' . j I* , I { ^ i 

- ^fi(llDX/s^-'-the-^Wiienca^haW ^been do.ded -on* an^ observation fc> IniFigtire 3.1* ^ ^ * 

Reading from left to ri^ht, the^context is drill. (1), they are using Cuisenaire 



rod^ as materd^ls (1), and the instructional, strategy Is questio^arid 



answer (1):. 



FIGURE 3.1. 



Obseirvaitlon Code Sheet, with an Illustration of a Possible Classroom Behavior Sequence 
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First teacher behavior (TB,):" "How many wJds will you take?" [Code line 1,. 

•columns 4 and 11: TB^ - question — > Indiyidual}." , ' * 

9 . * ^ 

First student behavior (SB"^) : "rwill talfe three i;ods." [Code' line" 1, 

* ♦ * * ' * 

column 12: SB- - answer]. ' • < " ' 

i • • . . ' . • ^ 

, ..i , • , . , , 

second teacher behavior (TB2):* "Good."" [Code lineal, eolumh 22: TB2 - positive 

..feedback] and signals that the student should repeat what he has said [Code 

line 1, column 28: TB2 - asks to repeat - H (nonverbal)]. 
SecWd student behavior (SB,): '"'1 will take three rods." [Code-line i, column 

37: SB- - answers]. * 

2 - , 

1 ^ J. 

teacher :• While the student is repeating ^the sentience, the teacher holds up 

a finger* for each -word. , When. he is finished she indicates by pushing the 
.first two fingers together that he cshould use the contraction. Since the 
interaction is still with the same student, the observer drogs down a line 
on the right-hand side of the sheet. [Code line 2, ^column 11'. TS>^- npnverbal 
proippt]. ^ ■ . ' * . , 

student: "I' .ill -take three rods." • [Code line 2, column 37: SB2 - answers]. 
.TeJ^%: "I'll take three ,rod;s .," ^ [Cpde^Jin^ ,3,. coliprv2^| Tj8.2_ 7^fo^dels]^a^^^ i 
writes contraction on boat4 [Code line 3, column 31: TB2 -.writes on board].,. 
Stiient? "I'll, take three rods. '''^ [Code- li.ne 3, column 39: .SB2 - practice]. 
Teacher: "O.K., everyone. £ll take three rods.." [Code line 4, columns 7 and 11 
TB^ - models class]. ' .. 

' students: "I'll take- three' rods." '[Code line 4;, columa 14: 'SB^.-.practicel. 
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j^t": '. '• - „. • Observer Trairiing , " ' • 

N ■ " ^ , / • • • • . 

* Using the videotapes of the 'center teachers, the associAf project , , 

<■ ^' ' ^ ■ ■ 

i director and research assistant trained /the coordinating teach'er and both 

^- ' ' ' . ^ ' - ^ ' ^ , • * - 

* tfe^ter-^bservers on the system* In-class M:raining was accomplished by placing 

I 

two observers in the 'Samexclassroom and comparing their interaction coides for 
agreements 'and disagreement* Those categories where 'the highest disagreement 

occurred were in several cases further modified and in others batter clarified* 

{.,•■• • V' ■■■ ' . , 

to improve Intfer-observer agreement. Once these final modifications were 

— » '- * . . 

completed, reliability studies were begun* ^ , \^ 



Observation Schedule 



Day School 



The coordinating^fceacher drew .up a schedule for ^Ssefvation^ which . : • V,J^ . 



L allowed for 'a 20-ininute observation of each class, on four different days 

' I ' ' i • ^ ^ . ' ^ , • ^ * . • • 

^ ' ' of' the week. ^ For those classe's meetihg only two or three times a w6ek, ^ 

' Observations were scheduled for each aeetiiig. While thfe coordinating teacher 

,was.».ofigfnally scheduled to observe in e.iach classroom; at least once a week ^ • 

/cbnmiitmeiits. ' She did, however ^ * 



k>"Vi^ i • fefi§ bd'came 'liipos^bl'^ due to other. job|c 

itv. occasiofnaily substitut"rfor on^p^^^eiii^t^ ^ ^. J 

t^,:\" \ ' (See tbe latter portion of. this ctiaptef on the reliability of th^ observations ^;)^ ^ 
|0 \ , T pbsBryation of the :d^y-sclio6l classes fiefah ^January r9th and continued through 

/ 'rf ' s'^erli factors affected the actual number of ^l)bserval:ions made; e.g., 
fr\"'^^y^' school K^idaygs -and closing as wei">/as teacher and observer illness. The ^, 



r nufliber of observations per class and observer are presented, in Table 3.1'. 



1 



^ J • . TABLE 3.1 . ; 

Number of ClassrooinSQJiservations: • By'' Class and Observer 



Ob.server 1 

' 8 
ft 9 
■ 1 . 

15 y 

•15 
15. 
• 17 • 
16 
16 

15 . 
15 
16 
14" 

188 



V 



5 
" 7 

5. 
5'' 
7 

. ,4 
-33 



Observer 2 
0 

. 1 

.0 
0 

0 • 
2 
0 
0 

• ■ ^1 

' 1 . 

1 

• p. 

0 
6 



3 
2 
4 
1 
3 



I \I4, 



UDserver j 
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29 
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' » ■ ■ i • 

.%J»te ^that'fdr ail but three of the classes, (two of whiich met only, two or-, 

" ^ . " * . ■ r . 

. three t±mes~a-week) > we have an average of 30 observation sessions *per class, 

. \ ' 1 , • 

giving us a ^ata base of some 600 minutes of observation per class, 

. Nighlf School , * • ; A , • ' ' 

A similar observation schedule was drawn up for the night school. * . * 

, . Here, howfev^i:, since class meetings were held for two hours, three times- 

a-week, one observer observed during the first half of a class and a second ' ^ 

observer observed during the" second half. Nightrschool observations w6re 

mad^by the same tester-observers and the coordinating teacher.. 'Observations' 

were mad^ at two different times during the semester: between February 5th 

and March 17.th, 1976, and again between May 3rd and May 25th, 1976. 

* - « ' ' I ♦ 

Again, the number of observations per 'class and observer are -listed in 
Table(^3.I. For Jthe si:i night-school classes we had aii average of twelve 
Qbservatibns per* class, giving us a data base of approximately 200 minutes 
^ per class J • . > • 

* " Processing of Observation Data ^ ' , ' ' ' 

• As Glas^rooqj observation data w^re being pqllected, jthe data wej:;e teihg ^ ^ 
o transferred, and coded onto forms which were sent to ETS." These dat^f were 

pxocessed and data sets werfe made fot use as input to the gomputer*^ , An 

^ ■ ' ^ o * / * ,1 ^ ' 

editing procedure similar to the one used for Student d^ta v/as tjsed on these 

d^a. . ^"^^ • . ' , ' * . 

^ Dajta for analyses we^e then oCreated^> and this process is .described in 

: - detail in the Section of* this chapter entitled; "Analysis of Behavioral^ " . 

<^ ^ , / Observations." "M^ny analyses including summary statistics, correlat|.ons * 

* * ' . - ' » . • * ' x 



. tod factor analyses were completed. Factor scores w^re then created for 
each teacher andwere placed on each student's record^^_(the- appropriate 
Student .and teacher were matched) for the^lnal analyses. Please see 

. Appendix D for the detail of^the data processing, * 

Desq^riptive Statistics Pertaining to the Observational Data' 

^ Table 3.2 gives the observation item numbers, category labels, xbdes*^ and - 

means for day and night-school classes.* The category means may be interpreted- 

as the proportion- of observajtion episode^ in whifh the e\^nt was observed ^ ' 

in the day-school or nightrschool classes • However, these will ^^dd to, 11 00 

/only when the- behaviors within a group are mutually exclusive «nd exhaustive. 

Note ' that the means f or *day and night-s.chqol classes in most instances^ 
, ^ ^ ^ , / ^ ' . • ^ 

ar^ highly similar, , Xhe categories where this is not the case* are: night- 

schp«tl teachers are more likeiy to work without materials (item 2); day-"school 

teacherfe-^re more likely to use ''corrective feedback (item 12), prompts (item 15) 

and theit a.etions are moreMikely tp b^ nonverbal (imn^ 23) ; night-school ' 

, ' ^ . ' " * - . * " ' ^' ^ 

students' Initial behavior is more likely to be ansvrering^ Sx xjuestioning ^ 

, - _ • * - 

(if em 9), while these behaviors^ are more likely to be successive behavior » 

for d^y-school students Xi^em ZO). . ' .-^^ ^ • ^ 

.Overall, the similarity between means fSr^the remaining categories ' 

, ' & ^ ' „ *■••■.**.. " 

''indicates that> similar amounts Inji the^ same kinds of behavior were ob^erv^ble ^ 
ill 'both the day and night-school classes,. A. detailed description of-how^,the 
observation^ .data w^re-^'anaiyzed folioWs, *^ . 

;i r ; jr- [ . ^ , % , , ^ • - * • 

^VtoCepiSode is defined as: sequence of behaviors between the teacher and 
'^.^any partlctdar/.^tudent. It may be ^initiated by either teacher or student 
and ends when Jthe^ teacher addresses or r^pbhds to another student. 



, • , . TABLE 3,2 . 

Item Categories for Classroom Observation 



Mtnber of 
Cat/gotlca 

• . 2 



Category Labels 



1 Context 



1 Drill 

2 Other * 



' r Codes 
OTHR 



Day School 
Mean3+ 



.99 
.01 



Night School 
• Means-Hj> 

1.00 
.00 





' w. ■>- » " 

2 .j!(aterriali' . * 
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r « 


,N0 MAILS 


.53 


— ^ 

.65 












' 2 


Rods s « 




RODS ' , 


» ^11 


.12 • 






• • 
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Cartoons 




^ CARTOONS 
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r 
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Pictures 




PICTURES 


^ -05 


.05 








\ ' ' 




5 


HIqgo Sheets 




MIMEO 


.23 


•17 












, 6 


Objects 




OBJBCTS 


.04 


.00 




• 




^ 7 


Sight Words 


SIGHTWDS 


.02 


• 00 









3' 


Strategy /Model > ' ^ 
• . .. 

<> 


5 


1 
-2 
3 

/4 
"5 


X^estion & Answers" 
Free Response 
Repetition 
Directed Dialogue 
Discussion 


FREE' RES 
REPEAT ' 
DiteCTED 
D^SCUSSK 


.94 
.03 
" .00 
.02 
* * .01' 


^ / \ — 

.99 
.01 

» .00 
•00 
- .00 


' — o\ 




*' > A 


Initial Teacher Behavior 


^2* ^ 
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Question * 


Jtbi-que's 


.32 


.40 
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* ' 5 . 


. Initial te&cher Behavior 




-s. 

2 


Models 


TBl-MODL 


.10^ \ 


.11 
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loi^^ial Teacher Behavior 




2 


Writes on Bo£^rd 


TBl-WOB 


.05* . 


.06 




* ' ^ 


7 


" Inttial 'Teacher Behavio)^ 


A 

m 


1' 
2 
3 
. 4 


Serial Redirect- . ^ 
Direction , 
Explanation 
pther . 


- ^ TBl-SR . 
TBl-DIR 
TBl-EXP 
TBl-OTHR 


..01 
. .18 > 
.04 

.03 * 


• .02 ^ * 
' .18 
.09 

.01 • * 


i . 






Object of Teacher Behavior 


\ 3' 


1. 
3- 


Class 
Group 

Individual ' • 


.CU^S - » 
. . GROUP. 

INDIVDL. 


.01 

*70 


.31 

.00 ' 
J>9 


> 



9 '^IniMal Student Behavior 

^ 

10^ Initial Student Behavior 
11 Initial Student Beh^lvior 



2 Answers - ' • " 

2 Practices . • , • . 

1 Free Response . f". 

2 Writes on Board 

3 Reads ' ' ' ^ 

4 Chooses Not, to Respohd/ 

5 Asks^^'estion ' ; 

6 Cbnvefcsation « ' 

7^ Student*»Student Feedback 

8 Other ' • < 



SBl-ANSR 
' ^SBl-PRAC 

SBl-FREE 

SBl-WdB- 
, SBl-mD 

SBl-diOT 
. SBl-AQ 

SBl-CONV 

sbI-sfbk. 

. SBl-OTHl^ 



.35 
» .11 

•-.01 
: .0^ 
/ .00 

.01 
.02 
.02 



,45. 
.11* 
•'.05 

r -01 

.06 

.03 • 

;i^4 ^ 
;o3.v 



70 



Item 



"12^ ^J^orrcctlvc Feedback 
13 Quality of Feeback 



•NuKber of 
Categories 



J. Category Labels 

2 Corrective 



OK 

Positive 
Negative 
Other 



Codes 

FB-CORR 

FB-OK 
FB-POS 
FB^-NEG 
FB-OTHR 



Day 
School 
Mcans-K 

.50 

.07. 
.05 

' ,oa 



Night 

Schoal 

Means-f 

.'30' 




1^ Successive Teacher Behavior 

15 Successive Teacher Behavior 

16 Successive Teacher Behavior 



2 Models 
2 Prompts 



17 Successive Teacher Behavior 



1 Asks Student to Repeat 

2 Repeats 

3 Explanation 

'4 Writes on Board 

5 Direction 

6 Alternate Response 

7 Other 

2 puestion'' 



TB2-M0DL 

TB2-PR0M 

TB2-ATR- 
TB2tREP 
TB2-E& 
TB2-W0B 
. TB2-DIR 
TB2-ALT 
TB2-0THR 

TB2rQUEs' 



.22 

.47 - 
• 16 

.07^ 

.06 
/.02 
.02 
.01 

.09 



.12 



18 Other 'Student Behavior 
19* Othef Student Beharvior 



2 Student Models 
/ 2 StudeAt Prompts 



SB2..SH0DL 
SBa-'SPROK 



.09 

^05 



.05 
.04 



20 Successive Student Behavior 

21 Successive Student Behavior 
'22 Successive Student Behavior 



2 Answers 

2 Writes on Board 

1 Free Response 

2 ^ Practices 

3 *. Reads . / 

4 Chooses not^to Respond • 

5 Asks Question 

6 Conversation 

7 Student^Student Feedback 

8 . btlijer . 



SB2-ANSR 

S^2-W0B^, 

SB2-FREE 
SB2-PRAC, 
SB2-READ . 
SB2-CN0T 
SB2-AQ 
SB2-C0NV 
SB2-^FBK - 
SB2-0THR 



.75 

.02 

.02 
.22 
.02 
.00 
.04 
.00 
.04 
.01 



.40 

.02 

.01 
.17 
.05 
• 00 
.01 
.00 
.03 
.00 




23 Observer Connents 



7 



f . z , 

General Cotoment Cl-GE^ILC 

Student Response in Spanish ^C2-SRSPN 

Teacher Action Non-verbal^ C3-TA-NV 

Backwards Buildup^.Exercise C4-BLDEX 

Teacher Reads * . * C5-REA6g 

^Xeapher.-Re8pqnse In^Sj^anish ^ - C6-TRSPN 
Comment on Back^^f Summarp'Sfieetr ' d7-0TtfR*^'^ 



.02 
.03 
.10 
.01 
.00 
.00 

;.or 



.02 ' 
.04 

.00 
.00 
.00 

-'~;04^ - 



Mean ntimber. of occurr^ces per episode. In most instances these may\b^ interpreted as proportion o 
eyent was observed.^ JThey will only add to* 1.00 when groups of behaviors are niutually exclusive and 



f episodes in 
exhaustive. 



whi<?h the ' 



Binary iteas ar/ de&ignated appropriately as having two categoriesi but only - the "behavior present" category is labeled 
• and scored for analysis. . , ^ ... 

**Some of the binaryv^tena which cpuld be repeated in rapid succession were simply. counted, instead of making a separate data 
entry' for, each 'in^wice. , ^ \ ' 



ANALYSIS OF BEHAVIORAL OBSERVATIONS 



The system* of recording cla'ssroom behaviors employed in this study 

yielded a 23-item response record for each episode of interaction 'observed 

in the classrpom. Each such episode coiiM be- initiated either hy the teacher 

* > ♦ * • 

or by a student, and could continue through an extended sequence of dyadic^ 

interchanges. Provision was also made for recording participation of d.ther 

students in the basic' pattern of teacher/student interactions. Every interchjfge 

in each sequence of interaction (episode) was initially recorded and coded 

Indiyidufilly; these iridividual instances were then aggregated summing to 

gpt one data-record per episode. An episode began at the initiation of any 

* 0 

new dtrteractiOn. e^.ther by the teacher or 'by a student. * t ' ' 

• * ^ . . - - C\ a 

Methods of Ofg^niz^jrfg Dyadic .Chains of Interaction 

As seen in .Table 3.2, sofae of the things a teacher could do^ to initiate an • 
interaction with a particular student, group of students,* or the class were: 
-asfc a question,- model *coj?rect juaage, jar give^dlrectiona (use of the Obseryation 
lexicon d.n Appendix G in conjunction with Table 3.2 will be helpful throughout 
this section). A student could initiate interaction by asKing a^'questton, 
for example/ or could continue an episode* of interactioi;i initiated by thfe * 
teacher by answering a question, following instructions (e.g., to read or ta 
write on' the board), practicing, etc. If the sequence of interaction continued, 
it could 'lead to further teacher behavior in the form of corrective feedback, 
modeling, prompting, questioning, ^etc. ; x;/hich could lead, in turn, to the 
student's' second attempt to answer correctly, to more practice, etc. Long . 
chains of- cyclical, dyadic interaction could thus be, and indeed were, coded 



s "successive" behaviors in each episode of class-room interaction. ^ 



' For purposes of data' analysis we chose to treat each distinct interaction 
sequence or cdassroom episode as the tasic unit. This means that all teacher 
and, student behaviorSv after the ini,tiation- of any particular Interaction 
sequence, were aggregated* We thus retained 'only the distinctiojci between 
initiating and subsequent behaviors/ By noting only the lei^^^h 6f each ,chain 
of subsequent interactions we f yewent the possibility pf dealing in any 
detailed fashion with sequential analjrsis of classroom interaction episodes, 
but we also avQid^d the problem of handling chains of varying length. ' By 
maintaining' the distinction between initiating behaviojrs and subsequent conse- 
quences, -howevet, .we left some opportunity for the most basic aspects of 
sequential organization among -classroom behaviors 'to emerge in our analysers. 

Had we not aggregated across successive interchanged within each dyadic 
classroomv^episode, we would have been left to deal with the original data in 
its basic categorical form; i.e., ^23 multicategory items as lisfed in Table 3.2. 
Each*of^the original behavioral interchanges was recorded as a 23 item "word" 
wi.th a total of 81 (2+7+5+2+2+2+4+3+2+2+8+2+4+2+2+7+2+2+2+2+2+^^ categories. 
Even if we neglect the fact that inore than one teacher oir student behavior can 
occur at once (e.g., modfel and* expl^ua^tion) , each iriteractiari recbrd could' 
represent any bne of 154,350,000 (2x7x5x7x3x10x5x10x3x10x7) possible different 
patterns of * interaction. We could thus s'umma^ze the^data from all individual 
* instances of interaction .in an eleven-way. contingency table. Such a taBie 
♦would be mostly empty, however, since only 20,000 instances .of behavioral 
interaction were observed during the entire 150 hours of cla^ssroom observation^ 
in this study. ^ 



* Even .our aggregated data must be viewed as having" some of the. features 
of multicategory data since many interaction chain*s wexe very shpri in 
length. Likewise, the catepx^ies in each of th^ 11 partitioned subsections 
of Table 3.2 are likely to be' mutually exclusive and exhaustive; for instance, 

the occurrence of one categqry of teacher behavior such "as. modeling generally 

■■ ' ' ' ' . • 

precludes the occurrence of certain other categories of teacher behavior. 

' o • ' ' ' ' ^ ' / ' 

There were many dependencies among the categories in each subsection of - 
Table 3^.2 which' had to be t^en into account In our analysis of the data just 

' . • ' y . ' . ■ ^ -> 

as though it were p,ure mul,ticategory .data. 



* Objectives of the Analysis of Classroom Interaction" * 

Our main interest in the analys'is^of the' behaviQjral obse^r^ation data , 

is to find, if they occur, fairly' stable and distinct patterns of classroom 

interaction. That is, we want to find out if certain teacher behaviors -give 

rise to or are associated withj certain specific student behaviors. The possibility 

of mqxe-or-less stereotyped chains of interaction is thus .being entertained, 

along with the po^rsibility that, these are^ characteristic of certain contexts, 

materials, or classroom groupings. From- the point of view of (analyzing 

' , ' - ' ' • 

categorical data we are interested iu^the patterns of association shown between 

* . ' • V . f V , , ' 

all pairs of multica|:egory items (subsectiomN)j ^iv en in Table 3.2; I.e., in 

all two-way faces of the eleven-vay contingency table afiuded to earlier. , 

Because categories of each item. tend to be mutually exclusive and exhaustive, 

we must only take account of associations betw^^" categories which belong to 

* ' *> • * * 

, different items in any analysis of association between teacher behavior. 



student behavior, context, etc* Our problem is not unlike that of avoiding, 
the contaminating influence of method variance^ in factor analysis of multitrait- 
multimethS*d correlation matrices, and our solution is equally applicable to 
that situation; i.e., simply 'do i;iot fit the mono-method submatrices. 

McDonald has suggested methods for ,the common factor analysis of 
raulticategory data (1969) as well as methods for conducting factor analysis^ 
*»w^th residual covariance matrices of prescribed structure (1970) which can " 
be applied td the data at hand.. His work grew_out of a long search^ for general 
exploratory methods for the analysis of multicategory data. Guttfnan (1941) 

initially proposed multivariate analysis of contingencies h Lazarsf eld (1950) ' 

0 * , ; X ^ 

dealt with general lalent class and latent structure models; Burt (1953) showed 

- ^ ^ 

Jthat xesults'equivelent to those, of Guttman could be obtained simply^ through 



principal components ^an^Iy sis of covarianc^ among item c^tegqries coded as 
binary vectoiVs. - , * 

McDonald (1969) proposed a complex weighting scheme to bridge th^ gap 
between Burt'^ principal components and the more desirable common factors. 
The latter, c^n be,, made very Simple, however ^ b^ ,^se of a flexible weighted 
least-squares method of f^tor analysis (Yates, 1971) which handles multi- 



category^ data simply by n(^t fitting, those covariances am9ng binary-coded item 



. categories^rwhich are biased due to mutual dependencies. Thus, in^ead of 

'just fitting the one-way marginals In <a contingency tabl^. (on the hypothesis i ^ 

of no interaction, as in the Pest of independence) or the alternative of 

fitting both one-way marginals and certain two-^ay faoes (as with a log-linear 

1 model), our approach compromises by fitting one-way marginals (means), while^ 
1 ' "'*" " . — . ^ 

1 approximating, to any desired degree of -tolerance function of dimensionalty) , 

1./ ■ 
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all of ^the two-way faces of an n-way contingency table. A mu}.tldliaenslonal- 
lilxiltiplica^ive model (factor analysis) is used to estimate the elements in 
a joint proportion matrix among all pairs of categories from different 
multlvcategory items in order to. accomplish the required fit* ^ - * 

^ ^ Methods of Analyzing* Classroom Interaction ' * -. * 

. !' \ ■ ' - • ' 

Since the multicategory data in this. study were partially aggregated 

^ ' . A • •, ' 

^ (within behavioral episodes, the chpseri units of analysis)., they could not 
:u3ed to generate a conventional joint proportion matrix. We therefore 
• applied common factor analysis with residual cqvariance matrices of pre- 
scribed structure (ignoring relationships between categories belpnging to 
, the lame item) to the matrix of product-moment correlations among, all 68 

.V , ; , — : ~ 

I binary-Qoded item categories listed in fable 3.2; Since we were now dealing 
^-with continuous data aggregated within episodi^s, we could make direct use of 
\'the counts^^^ rapidly recurring events, as well as of a measure of the duration 

of "those interactions which occupied'^more than cfne minute. » / 
^. ' -The availability of data, on mar^y inHi^dual ,behavio^.^l. episO-d^s made., 
J it possibl^e to conduct factor analyses -of a large number' of binary var|.ables ^ 
.separately . for the day-school and night-school samples, ^en though only six ^ 
teachers were studied in each sample. Several different^ methods were used to' 
determine how many common factors Should be extracted from .each correlation^ 
-mafrix,^ If anythiAg, we probab11y~erred. on the sidfe of extracting too many 
factors "becau^ of our desire to avoid missing tny possible basis for dis- 
tinguishing between teachers, fable 3.3 presents ^genvalues of the original ^ 
day and fiiglit-school coTrelatJ.on matrices using highest^^f f-diagonal corre-^ 
' lations as the initial communality estimates. ; 

r * 



TABLE 3.-3 



Variance" of the First^lZ Principal Axis Factors in{ a Preliminary' 
Analysis of Correlat:^pu Matrices Derived from Observational Data ^ 



Day School ^ . 

Cumulative 
EiRehvalue s>r, / % Trace 



A- 





/I 


2.53 


36 


1.77 


46 






1.54 


55 


1.49 


64 


f.33 


■ 72 


1.26 


79 


1.09 


' 85 


.86 


90 


.66. 


, 94 


.62 


98 










: .58 






Eigenvalue 

"3.60 
'• - 2.71 

2.17 

2.03 

1.88 

1.-44 

i.3q_' 

1.15 
.81 



Night School ' 

Cumulative 
% Trace 



.72 

.58^ 



19 

) 33 

45 



56 
.r. 



66 

73;-, 
80 , 
86 ' 
91 



94 



98 



100 



' 1 
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As can be seen from Table 3,3, there is a ^'suggestive l?reak in the 
distribution prf eigenvarue^ .after eight -factors for the Aight-school sample, 
but we retained nine in both samples to maintain comparability. After 
..weighted least-squares (minimum residuals) fitting, the nine-factor root- 
mean-squared residual correlation between binaty-coded categories from 
, independent items had been reduce?d to ♦027» for the day-school sample and 
• .032 for tlie night-school sample; ^ * li 

The nine minimum residuals (MINRES) factors for each sample were 
transformed po oblique simple structure using the direct GEOMIN hyp'erplaftie 
search method (Yates, 1974)* The resulting primairy factor pattern matrices 
*and primary factor inter correlation matrices are presented in Appendix E. 
Because the two samples used ^provide an opportunity to compare factors we 
will disQUss the results from, that perspective* a 

< 

Factors of Classroom Interaction 

.One ,Ppji4it to'" note about the factor pattettis presented in Appendix E, 
^rt^"^b£es^l and 3, is the fact that virtually all of the factors oWtained are 
truly factors of classroom |Lnteraction ;. i.e*", cate^rie3 of behavior from 
distinct and independent sections of Table 3.2^ load on each^factor. This ^ 
is as we would^ like it and is due to the fact ' that correlations among 
mutually exclusive and exhaustive behavior categories were ignored in the 
MBIRES fitting phase of the analysis. One minor disappointing feature of 



i 



the gutcomesllk^that *one factor' in each ajialysis represents a major contrast 



between t^c^BKereat modes of classroom interaction. This result may say 



*S3omething* valid .about the organization of classroom behavior, but it compli 
cates matters, f^om the point of view of discussion and interpretation. 
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Rather than discussing these factors In a rigid but superficial way 
with respect to the relative amount of variance accounted for, size of 
loadings," etc • , ]je prefer^to try to convey to the reader the outcome of 
our own attempt to inf^r how classroom behavior seems to be organized. 
After all, the point of un<Jer taking these analyses is not t'o generat;e 
>. numbers that we are then compelled to tjake seriously, but to help us foirm 
- a conceptual framework bas'ed upon empirical results through which we can come 
to understand aiid discuss classroom, interaction. Thus the reader who is riot 
excited by perusing tables of numbers and who trusts our judgment^ nfeed not 
worry about either the Humberts in Tables 1 through 4 of Appendix E or about 
, the intri^cies of factor analysis* . . ^ . . 

Comparison- ofj^ay School arid Night School , in . 
Patterns of Inlti*ating Classroom Interaction 

Although there are/many similarities between the patterns of classroom ♦ 

interaction identified" by factoring day-school and night-school data, it is 

' ' equally informative to note some of the major dif ferences^^^ It seems that a 

major source' of discrepancy between- these samples (or analyses) is related 

to the sequencing of classroom .behavior. ' . " , 

Teacher-model-student practice factors , ^e ^ound "teacher model"- 

"student practice" factors in both samples. Ift. the day-school analysis 

separate "modie^l-fhractice" factors break out for the* initiation (V) and ^ 

x followL-up. (II) phases oi interaction. But in the night-school analysis 

.one factor (X) involves both phasSs of/l^itey^ac'tlon, while the other (V) is 

^ r * . / V/ ^ . r--^ . 

, ^ ^ ' £^ ^ jt. 

^The Rbman numeral^l4 parentheses [refer to the factor columns in t-he ^ 
respective tables. Note that the factors are printed in arbitrary order 
,but are seqiience-numbereji in ftqpms of their relative sizes.* 

" . • . ^ 
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for follow-up . The distinction'between phases of the "model-practiiM* 
interaotion pattern in' the day-school analysis* is further accentuated by 
some indication that initiation of the sequence le^ds to a request to X ,£a. 
repeat the response rather than^^to^^automatic continuation of the "model- 
practice" seqiipnca.- 

. ■ Differences such as J:hese could be largely influenced by .the particular 
***teachers invplved in^y and night-school an9,lyses, but, 'if so, they ar^^e 
to differences in the struct^iring of interaction from moment- to-mqment 
(epis(^de-to-epiSode) and not to differences in the .mean amount of "model- 
practice" invoked. Likewise, we \cnbw that the factors which emerged from 
our analyses are not due to any large extent to dif^rences bets/een teachers 
^thin either sample, because essentially the same factors were fourid even 
after partialling out^( removing) t^ach^r differences. 

Teacher, direct-student read factor * tike "model-praotice", another 
major pat tem-tJf classroom interaction which seems to display differential 
sequential organization in different samples is "teacher difect" - "student 
read". In the'night school, again (coincidentally.?) , we find that initiation 
of this sequence (IX) deads to its .continued expires sion or repetition— bub 
.the teacher may "^sk the^ student to jfepeat" in^oijtinuation of the reading 
sequence. This suggests rather sustained, continued reading sequences uil4er 
'the teacher's direction,' with repetiti((n as necessary*; In the'day ffchdoi^ 



0 K. 



on the other hand, factor (IX) suggests ^ pdtftem of initial "teacher' direct 



Student read" behavior but,; if anything, a low tendency for furtHer teacher 
direction or explanation; i.e., we see a short .residing episode 'which i^.no*t 



continued* With "teacher direct-student read'^ we find, a contrast, between . 

. ' ... . ''^'^ ^ ^ ' ^ ^ 

xilassifoom interaction patterns: since- something.. similar happens- fot -both ' 

day and night s.chobls, however^ .perhaps we should take, it seriously. 

Teacher .direct-st udent re^dr or ask question factor . In addition 
— " ■> - - - — " 

to\what wq^ said immediately above abouf the "diifett-read" sequeiy:e_in each 
sample, ' there se^is to be another sequence (VII day, VII nigHt) of ^teacher 
direct" - "student read and/or ask question." This behavior pattern seems 
-to be in contrast- to one of "teacher ask question" - "student ^answerr," 
TherQ is the further complication of a ciass 3s. individual distinction in 

the nigfit school* There "teacWr ask question" applies 'to the class-as-a- 

> ' a V' , ' 

^ * i • 

whole .^whereas individual students are "directed to rfead/* One way of 

interpreting what is going on here is to regard this factor as a contrast 

between two situations which can alternate withiu the same classroom during 

any given se'ssion of observation; we will pursue ^this interpretation later 

'in this chapter*' In the oii^ht-school sample there is some slight suggestion 

^ ^ m ' ' , , ^ -» 

that students may be reading mimeo matetials but are bein^||^sked to answer 

quest lon&^ about pictiures. In the day sqhool there is some?, indication that 

'the teacher^ sNbehavfpr after a student^s attempt to rfead ,or atftet asking a . 

question is an kxplaiiation; this, in turn,, may be followedfby mo|^e reading 

and question-asking 6n the^^patt of the student— -a rather tutorial^' setup. 

Teacher questd^on-student answe^C factor* . Jn each sample there 'Is also ^ 

' * , . 6 

another, distinct, "teacher questW'*r-"student a^iswer" factor (III day, ^ 

I' 

IV night) that is not contra&'ted with student (question-asking, Just as we 
already saw distinct "direct-re&dl' factors^^.in each sample-. In both samples 



it is clear- that *this som^What more ."^ure/^'cjiiestion-answer" paradigm is 

- '\ " *^ . * ^ V ' ^ 

* directed at individual students rather than^ti?* ^he class'at large^ In 

both samples there is some indication that the instructor might subse- 

. ^ ' < . r . * 

. *\quently repeat- the", question." 

Therfe are a few., other suhtleti^es regarding the '^teacher-question — ^ 

"s-tu'depi answer" factoit, especially in the day school. A pattern of some 

positive feedback is suggested fox both si&ples, and iirtere^tingly, a 

• > - y ^ . . ' * # \ 

disinclination of ^ the instructor to be writirig on the^boax^s while asking 
. ' . . ^ * " * y ' / 

questions ,XDf an individual* This latter pattern prpbably relates to the 

• obvious utility of the blackboaird for communicating to the, entire class 
r^theif than ro individuals. In. the day Ischool this writing on the board 
(vs.,quefetioning an, individual) se;ems to take time dhd to ' go , along with 
modelipg or explanation— with subsequent student practice, conversation 
and otr\€5r behaviors. ^ . ^ * ' *x ^ 

Let us now briefly ,nanie and codify the four factor? introduced above, , 
before moving on to some^^of ^the major dimensions ^of clas 

whilh seem 'ta folloV these initiating e^i^tst (1) "teacher mo.del"--"stud|nt^_^, ' 
- .practide"; (2) "UacheV direct:"-" studeiit read"; ,<'3) "le.acher '.direct"^ ^ ■ 
"individual student read, or ask question," .vsf "teacher asks, questioti - student ; 



in cla^s answerk"; (4)- 'Heacher ^questions ,ihdiiidual"-"individual' student 



Cf .11 



{ / ailswers r. i 



" . ; Patterns'- of pia)33roo^* Int-eradtioi?. ^ ^ - • ^ * . . , . ' 

' ■ ; Whidh .-'ar4r Sustained. Once Initialed , ' ' ^ ^ ^ • 

, ;.' *The "modiel-practice" pattern *'seems fco l^^d to subsequent cycling in^the 



1^ / 



oighe §chool, just as/for "dircct^read^* there,* so we^have^ already had"* a glimpse*; ^ 
K of some of'' the sul^sequent or f6ir<jw-up behlvior pattfernS.* Th6re are also ^ever^l 



of- a "free response" stems from the teacher's having ind|.cated only 

' » * • ^ I* ^ 

the -structure and not the idea or vocabulary t6 be used .(there is^n i 
this pattern a very weak suggestion of teacher modeling to initiate, 
with writing on 'the board as^ollow-up) 

Comparison of Day School and Nig^Tt School in • ' • 

' Patterns of Follow-Up Classroom Ititeiraction ' ^ 

As for the clearly "subsequent" patterns of interaction detected, 
we see two majot patterns* connected with "corrective feedback." * 

Corrective feedback-model-practice factor . Thg first factor mentioned 
in this 4isc!ussion was "model-practice^* and we stated ther^ that it "breaks up 
into an initiating and subsequent factor in the day school, but appears as 

• . ' ■ • ' 

a cycle of^ continued interaction in the night school. We can now r-eveal, 

rhoweVer^ that something more subtle th^n a simple splitting ''up or segmenting 

t ' " ' ' ** ' ^ 

of behaviors is going on, since a 'corrective feedback-" — "teacher ^model" — 

f ^ ( ' » 

/'student practice^* pattern of subsequent interaction was isol'ated in bcji^th 
s^msles (4ay II, night 47). Furthermore, the presence, of slight, but cOnsist'ait 

• ■ ^ ■ . . ■ y ' 

loadings. su&gests that, this pattern of cor;crective feedback and modeling 

* t * . . ... * ^ 

^ight yell be associated with a prior attempt on the 'part of the student to- 

' ' 'I' * ' y r ' • " ^ ' ' • ' \ ' * 

r^ad something under the direction of t^ teacher. ' i 

** / ^ * ' " ^ \ 

Furt:faer, iji.th^ nigljt school. there is some indication tha*t other students 

may. pl^y a 'role .in'^mq^deling the behavior required. In the day school there 

is some" suggestion that the teacher mi^ht; ask the st:u<ient.to repeat ,the 

initial response; observer spnrments also indicate use of a backward buildup 
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Corrective feedback-prompt-ansver factor .- Anothet, quite distind^ pattern 
of follow-up interaction involving corrective feedback was recovered im both - 
samples (I day. III night). In this 'case ther6 is some reason to believe 
that the feedback is encouraging (especi^ly from, the day-school data) and 
the sequence goes: "corrective feedback"-" teacher ^prompt or ask to repeat; - 
'^student answer." sTnce prompting or cueing the student "on how^ to modify 
his or her response to make it correct predominates here-r-w^ will call^this*^ 
factor '"CF-prompt-answer." ^ 

However, there are other possible teacher reactions in addition to ^ 
the ones mentioned^ above; from the night-school data we see that the teacher' 
might Jturt her direct the student; in the day school ve see the possibility *- 
of questioning the students .^n either ca'se, the observer comment indicates ^ 



a strong tendency toward a nonverbal teacher actfi9P, especially in the day 
school. Here 'also we see the possiMlity that otKer students,^iiji4!i**get involved 
in modeling, prompting, or student-student feedbacjc (day scbooli. * In the t 
Ttight school Ve see some indicatiqn that this particular pattern of inter- 
action'^ may occur while the student is writing at the board. . 

Teacher question-student janfewer .f actor ♦ This description completes our 

* • 

discussion of the classrobm interaction factor' patterns except for one extra 
"subsequent", behavior ^Dund .only in the niiht^school sample: "^te^cher qjiestiop - 
Vstudeafi answer" (vill). This facfcqt is relatively highly (Sorr,elated , < i 26) 
with the "CF-prompt-answ^4i factdr' just discu'sse4, but "teacher' question" 
is .Included on that factor for. the day 'school. Retaining one too many factors 

. . w \y " ' \ 

in the night school cOuld accoun^Vf or such splitting apart, eepeqially since 



1 

■r. 



ERIC 




this factpjr lia^^gh correlations with several others. One of these other 
correlated factor^Cr p V20) is the initiating "t;eacher quest ion*'-"l,ndividiial 
stjudenty^swer" factor ^ tlV), so again ^he re may*' be a tendency for cyclical/ 
rep^'ated patterns of interaction in 'the night schooi. 

It is noteworthy at this goint that tfJe highest correlation found 
between factors *(. 37) occurred for the day school between "CF-prompt-miswer*' 
(I) and "teacher ^question individual" - '''individual student answet" (III). 
Here, again, we, se^ an association 'between questioning and prompting. Were 
one to take these correlations among factors seriously, it can l)e 'seefi that 
they would .le^ to a higher' order model of what is going on J.n the' classroom. 
Unfortur^tely, correlations among yfactors can be rather unstable so we 
hesitate to interpret them here. They have nevertheless been iivtluded in 
Appendix E, Tables 2'and^4 for the sak^e of completeness; the interested 
/reader shoiild interpret them cautiously. . - ^ • 



■' Classroom Differences in l^tse^action PatJ/erns 



Once the factor analyses of behaviorai obsei::y^tions"were completjed, 

we were in a position to investigate differences in classroom behavior 

' patterns— Ihe ultimate aim being, of course, tSJ, identify intei^esting'jion- ■ 

-Etkses arnqng- classropms. whici might accouht for kiff ererices -irf-'sfcudidt" - 
. " ^ . '•'.,">'• ' ''',•'.'•! • ' . ■ •• '"• ' 

^ aphifevement. We~ were interested in betwe.en-classr^om variation in ■ 

• ■ . ■ - ' ♦ ' ' • " . ■ ' r ■ ■ 
behavior for its owft sake, 4iowever, and therefore sought a tecTlnique l^hich 

- ^ . ' . • .".'!■■,' 

would" ,perfflit up to ■ optimally disc?rimina.te betwfeen j:iassrooms -ou the fesis - 



I, 
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We' settled upon the individual classroom as the focus of study at this 
stage because our interest centered upon 5:eacher dif,ferences as well as 
upon differences in the ways in which any given teacher might* approach 
Students of various ab:^lity levels. For'purpos^s of assessing variation 
within classrooms (i-e., the interaction of a particular instructor with a 

1 • ^ ' " " • " < * . ' I ' 

more-ot-less homogeneous grpup'of students.) we chose individual sessions of 
observation as the^ basic units of analysis. That is, we, contrasted overall mean ^ 
differences between classrooms on ^ach pattern of "Interactidn (factor) with 
the amount of day-to-da^ variation observed within classrooms for that pattern 
of interaction. This was done simultaneoilsly for* Scores on all nine factors, .^^- 
separately for tfie day-school and nigi^-school samples, ^ 
') ^ " Specifically, for the technically iitclined, factor scores wejre computed 

using. the complete regression method for each individual episode of classroom \ 
• inceractiJh- Mean scores were then obtained for each session of observation 
•fa3i5ro«itt:atelv,20 et^isodas per session) and these records weire entered into « 
a canonical discriminant function analysis in order to find' successive ; ^ ^ 



V orthogonal line4r 'combinations of tha^actor scores whith maximally ;| 
•dilcriiainated among classrooms relative to dayr-to-day variation, . , ; ^ . . 

Univariate- Analyses of Classroom, . , , ^ [ ,r-^- / \ 

V ; Differences ln> Inte/action Fattqj^s . * / . * 

; ^ , . As^a preliminary to diaqjission of th^ |Jis.criminan| fundt^on anal^s^s,' , * 

.:' / r " " h • / - • * > • » . ^ • • - 5 . .V 

\ let us^ake^^^'irief look at univariate, analyses of variance betwe^n.^ classrooms ; K 



';/^',^usinJ*sess;LDn ifleans as the basic unit's analysis. -Of course, thertf-is a 

■ (V* ' ■ ' ' - * ''i " . ' ' ■ 

-•V'- sepa'rate analysis of variance for Score's on each of the nine classroom 

- ' - ''int^r^ctioh 'factors discusseS earlier in each sample. These analyses a:re . 

: , ■ « ' '• - : rX J. ■ "■ 

;, . ' sitfrnaaiized in -Table 3.4 in terms ,of ^F-ratrofs aftrd^o-BaSfe-ties foi: evalu^n_g_^ ^T^^; 

. ! .the null hjBpo thesis^ of no classraam dlf f eretice^-— ^ ^ \ ^ - ^ <-:ifr 

•/ V * ' . * i ' ^ i ' ' . ^- • ^ ■ " - '^j^ 
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Univariate Analysis of Variance Tests for 
Classroom Differences on Factor- Scotes 



Factor Laoel 



"Model-Ptactice" 



^ "Model-Practice-Model-Practice" 

• "Direct-Read" ^ A 

^ "Direct-Read" - T 

"QA- Individual" vs. "Direct-Read/AQ" 
"QA- Individual" vs. "Direct-Read/AQ" 

1 "QA-Individual" * ' . . 

"QA-Individual^ 

"Student Feedbacjjl' 
"Student Feedback'' 

••other"* 
"Other" 

"Free Refpons 

"Cfe-ModB5?p4cfrrce" : ^ 



/l ^-Pro tnpt-AiisgBf" V^'" ' 
^^F-Proinpt-i^%r."„'^ ' 



Day School 
F-Ratio 

4.Q5 



4:^32 



2.33* 



17. AO 



7.70 



1.73 



Sf^J'Lc^^estibn-Mswer^^^ " f ' 

^etwaem:Ciassr:<3i6ias- Degrees of -^Treedom- 



ftithin'dlassrooms Degrees of Freedom 



7 



13 
.367 



Night' Schopl 
• F-Ratio 



i 



,83 



,63 



1.24 



4.81 



2.10 



1.20 




1.60 



** 



' 4.84 

3.40 

^ '5 ' ■ 
44 • 



*p< .05 '"" 



'f ' . ' QA: Teacher ^Juestion-Student Answer 
^Q: Student Asks puestion 
■ CF: . Cprrective* Feedbaifk 



J • 
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We will not .undertake a detailed discussion of the univariate analysis * ♦ 

of varianee results presented /in Table 3*4 until aftJOf we have completed 

discussion of the multivariate analysis (canonical discriminant function 

analysis)* The lJt%er will prov^e an overall test of ' classroom differences' - 

wh±cl|.. takes int^ account any dependencies ainoijg various classroom interacff&n ^ 

fap^ors. ..The results* in. Table 3.4 are presented here, mainly to justify our 

emphasis, upon analyzing the' day-school data in what follows; and to give the "i 

reader a* familiar frame of reference -prior to undertaking discussion of the 

multivariate analysfs. * 

f ^ • * • - ^ ^ ' ^ 

From the summary results in Table 3.4 it cafi be seen that many classroom 

• % > / / . 

differences were ^detected in the day-schoolUsainple but, presumably due in part 

• C/ ' ' 

to the smaller niimber^df observation sessions per ^lassroom, few differences ^ , 

were detected ^n the nigh^t-schdol sample ^ There were fi;om 15 to 33 observation 

/ < ' - ^ 4 ( ' / ' ^ ^ . * 

sessions per classroom in the day kcjiool cpmpared to from 9 to 14 such sessions ' ' ^ 

p^r classroom in the night school/ • . " ^> ^ * ' ■* ^. * 

, V • • • •/ * ^ ^ - 

,Ses«ion~to-Session Covariation ' ^ , -j 

^ of Classroom Interactioii.'Pattems • ' • - ' ^ . . 

Because of the greater precision of., day-school, classroom comparisons, we ^ 

will focus there in the following discussion ^ith only a brief discussion later 

' - ' ^ .7^ 

of the nig\t=£chool rejtlts. In^j::onsidering the univ«4sfte analysis of variance ' ^ 
results given in Table 3.4, it must be remembered ^at the'various classrqom * . 
interactjLon factors are .not uncorr^lated; thej^ore, th&re is apt ^e some . 
overlap or redundancy in the F-test results reported. In or^r to get a • .-.w .^'S 

feeling for 'this a^ociation between Iclassroom .interaction faotofs as tliey 
vary frdtn dayi^o day (obse^j/ati-on session'ta qbaeryation session) we can^ ^ ^ 

exasiine the pooled wfi-thia-^lass^^ms correlatida matrix jaiuon'g^actor jrnre 
session*inear8s pr^sent-^ in^Table'S.S^^^^ . * : • ..>~^ . 



/2' 



■ ■ ; TABLE 3.5 ; 

• * - ■ « * 

■•"Pooled Withiu Day School Classrooms Correlations Among ^ Factor Seofa Session Means 
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* ' Foi; purposes *of disciission we have ordered the rows and columns of the 
pooled wi thin-classroom cotrelation matrix in Table 3.5 so as t» biest reveal 

, ' * ' ^ . ' ' . . ' : 

tTie clustering of classroom interaction^ patterns. .Npt;e that most 'of the 
positive intercorrelat;ions are near the maiil diagonal and that several 
Clusters ^of associated interaction patterns have been blocked *off thd^ 
Note in Table 3^5' a ^tendency for the "corrective feedback-prompt-answer" 
interaction pattern to occx\r on those days vhen 'the "question-answer" pattern 
is applied to individual, students. These two factor^s thus go together to form^ 
a "macro" .pattern wliich links individual episodes, of clasB*^room interaction. 
Note alio that the associations present^^in Tabled. 5 are 4ue only j:o day- 



to-day fluctuation ixi' agtivity^pattems within each classroom — they are~Het 

'•'-*'*'. - ■ » • ' 

influenced by-^diff^^enoes in tbe overall mean level of classroom or teacher 

behavior^., since the la^'er Mif fe^nc^s iiave been ^removed (partialled out) V 

the pooling process.' We will, turn to between-classroom differences once 

the wit^in-cIassroonirVorganization of interaction patterns is clarified. 

A second cluste^^whieh emerges, has as its focus the '"teacher model" — 

"•student practice!' .interact ion pattern. The inclusion of "corrective feedback 

"teacher model"-"stud^^rpr^c1;ice** as well as "other" in this clustpr indicate 

day-io-day .covariation inn^t|\^^ repetitive, rehearsal-based sequences of ^ 

Interaction. This cluster =3f^%nked to "teacfher dire^P"-" student read" 

■ I . _ • • ;■</■ •■ ^ - " ■ ; 1 

through the* iatter'*^^s'correla£ibn wl.th "?teacher model"--" student practice" t-21) 

this, day-to-day. IssfcLation-Q^ "dire'qt-read'.' with "model'-practice" contrasts 

with the ■ten4ency af th& ftfrme?'»ho't to be §een on days when a "question- 

answer" (-.24) strategy is tin .effect. / ^*^,^„ *^ , 



f ,1 



^ It is interesting to^fits^e that the bipo^fic factor which'^ontrast^ 
teacher qliest/ionijig witji student question-'-asking (under teacher directions 




to read and ask questions, vs. DIR) is relatively inde^iendent of the other-^ 
~ interaction' patterns as^they vaj^y^frbm day » to day within classrooms^ Finally, 
we see that the more fltxible and/open pattj^rits of classroom interaction*- 
("student-student feedba^plSy**^ response") show a slight tendency to prevail 

(.13) but.^r^nWsi^tent'Vith ttve "direct-read" (-.25)/ . 



T 

."model-prSctice" t^^>2;i)/"othier'\ (-. 25^ syn,drome. . ■ 

' r . " ° ^ • ' . 

Multivariate Analysis of ^lasaroom . , ^ 

Differences in Interaction 'fat terns* « ^ ^ 
j — ' ■ — . ? ^ . 

Moving now.. to a conside'^gf ion of "overall^ classroom dif fef^etjces ^veraged 
across days of observation we must cons^er the outcome of c'anonical discriminant 

\ ' ^ * ^ / 1" ' " ; 

function analysis'. * This -analssiA Is designed -to identify fhose linear combinations 

' . ^ ' ■ V ^ ' " • i ' : 'L . 

of observed scores '&n Lhis case, classrpom inl|eraction factor* §cores)^Whicb:^ -"^ 
maximally" discriminate between. &roup^elativ6 .to wi thin-group variation. As \^ 
far as this study is concerned, the rpie of cimonllcal .disdriminant f.iincti4h 
arralysds is to identify ways in vhich classroom interaction- pa*t terns can i)^ 

' ' ' * ' » fL'* ' ' ' ' * ' ' * ' * 

'combined to yield stabte overall jsoffitrasts among ^he tclassrooms^studied; 'i#e. ,^ 
to -find how 'the dlassrooms. differ most claaYly from pn^ another oit the -average^ 



f 

wliile di^laying. minimal day-fo-da/ fluctuations. 



-'Table 3*6 summarizes the caVionical discriminant function' analysis 'results 
for -the day-schooi behaviorai observation data. Optimal discrimination was 



between 14 classrooms on the basis of nine factor scores, using obser- 
Vation* session means as the basic units^of a'nalysis* - , ^ 

An inspection of the results of the canonical discriminant function 

•V. * . * 

analysis among day-S5ii^aL classrooms summarized in Table 3* 6^ reveals that 
theri are two majar dimensions of variation between cljassrooma and two or * 




TABLE 3.6 



Day-School Glassiroqp Observation Data\ 
/ earionical Dl9crliiiin4it^ Function ^naiysiX^ 

Discriminant Function 

1 ' • * 

J 2 ^ 9 

.' ' 3 

5 

7 ! . 

WUk'^ A cjiter]L9n^=^-/L^2 for F 

Test for residual aftet removi^ 

Ftr^t di^ctitenan^.filhction 

First twlr^iscrimitiant functip^s 

First three disprimln^nt * ftmct^oits 

FirsJ: four dk"S<n:iminant functio 

First five djiscrimlnant .functions' 

^ First six ■di^scriminant 'functions 

.* , First seven ^discriminant, functions 




'Diseriminant Function ^Weights!j; Scaled 
'\Withiri-Gi^oups Variance on driginal 

Discrirainant Function 
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-.009"" 


• -.053 


-. 031 - 


; .248- 







three* more' minor dime^^ons. We^ do'tiat, howevei?, find interpretation of 
the discriminant function weigttt^ a?^ theV are' given i?*^ Table 3.6 either 
Straight-forward, or enlightening, - W ordar to gain moVe insight- into the 
*ideaning of these results^, fth^efore^. we Ijave elected fo. ccia3id-Qr tWe^between- 

1 gr6ups correlations between 'the original Scores .an<| the discriminant axes-. ^ 

\ ' - , * * . . ^ ^ t* 

These cprrelations are readily obtained a^d *have the advantage that they 

^ can be treated much like factor loadings 'of observed variables on orthogonal 

■* ' * < i > 

factoj ^.kes — the canonical .dfscriminant functions (Clif ^ an(i Krus/ 3b976), 
Because the fir^t two canonical^discriminant functions account for 

the majority (78 percent^ of all differences between day-school classrooms, . 

. \ -^^^ V ' ^ ^ . . • ' 

in tarms of stable patterns of 'teather-stjudent interaction, ,we have chosen , , 

to plot the classroom grpup centroids ,on these atjces in figure 3^2u In the 

or 



• figure each classrodm fs identified by a letter-nujubeor do^^^atioh. The . * ' 
initial letters range from A through F*and ident.jLfy ,the sl^ da^-^school ^nstruc- / 
tors. The numbers ran|;'e from 1 to 3 and, refer to pSoficiemcy lev6l of . the 



^ cl^ss being taught as measui;ed by the John Test. Ofhe lower case, letters 
.identify dif f erent- classfe At any given "'level' where they.Sre taught l^th^ 



sam^ tether* 



In a'dd:Msion to classroopiTgro^p .c^ntrcids,* wi| have drawn vector? In 
Figure 3.2 to represent thejway.,^e nine original factors, of momep^aty class- . 
room interaction project into the dfscriminant spacer Note that the important 
features of .these vecrtors^are their directions and theit Jtrelative lengths — . 
their absolute lengtris are arbitrary and^ave beed sc^lie^merely for convenierfce 
of plotting. «y visualizing the direct perpendicu'laf prg^ectio^ of classroom 



^The origin of Figur 



re ^3.2 ^s loc-ate?d-^^ tl&^cenj:efeO.£j^itv .(centroidf 
Ql all t?he. classes. . ' 4 ' . f • »- ^#^4^5*: 
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Plot of Day-School. ClassroorS Centro ids Classroom Int^action Factors 
in- the Space Defined by the Two Largefet Canonical Discrimiantj Axes 




eentroids onto these vectors (extended through the origin if necessary) one ' 
can get a feeling for the relative or:dering among classrooms in ^efms of the* * 



behavior under consideration, noticing the relative collinearity or 
Plgrpendicularity of these vectors ^e can get a 'feeling for hqw classroom 
behaviors associate or disassociate 'in the profiles whicH disi^ingiiish ^'tilass- 

rooms* from one another ^ . . - ' 

' . * ' - *, ' ^ 

As for the classroom eentroids"" plotted in Figure 3.2, bh^ most outstanding 

* ' - ' ^ j( ^ 

feature is the obvious clustering of classrooms taught by the same teacher*/" 

Thi^ phenomenon is partipiflarly striking in yiew of the fact that no 

information about who taught what class eijtered into any of the analyses 
leading to these results. The evidence is thus incontrovertible that. 
, teachers have consistent and distinct "styles" of interaction with students — 
Styles which do not in gene^fal vary^ markedly even when teaching classes of 



^jquite dif fepnt initial ability level . 



Only one teacher, C, appears to emjfloy /^idely dif £eren|:, strategies when 



t " 



leaching ^udents of. differing ability levels; however, it must be pointed 
out that no other teacher had the opportunity to teach classes (jontainihg 
students of such widely different ability levels. 

Because composite variables in canonical form are noto^q^sly difficult 
and dangerous to interpret we ^Will not attempt to label t^ axe? in Figure-:^ 
We can note, however, that the first (horizontal) axis i^S a contrast between 
^bout the same -clusters of variab,les that Le Id'entif ieil in the pooled within- 



S.SV^Th 



classrooms correlation, matrix in Table S.SV^That is/ a general patterrrof 
DIR READ, CF-M-P, MOD ftc', and- OTHER is cojit~rastp(/wlth QA IN.D and CF-P-A. 
The "l^etween-grouC^ correlations of the orlgi^l factor score variables 



]er|c 



with the cahonical , ax^ can serve as a basis for/ transforming (rotating) the 

lative ^nd interpretable 

^' IN 
7 



canonical vairiates i)it& aft more theoretically ir' 
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position* For this purpose we chose to use only the first four, discriminant 

axes since they aiccount for most (93 percent) of Uie variation between groups 

2 ' 

and little Significant variation remains (x = 59.4; 45 df) after the fpurtli 

' •* , ' . ' ~ " 

axis.is /considered.- Table 3»7 presents the DIRECT GEpMIN transformed between- 

classrooms canonical-yariate factpx. pat tern matrix, the factor correlation 

matrix, and "thfe factor structure matrix* , (• ^ * 

* Interpretation of ^Transformed Discriminant Function Axes * ' * 

yppjv transformation of the first f oui^ 'between-classrooms canonical 
variates we came. up with two major contrasts between t4acher-student inter- . 

action patterns (still accounting for the majority of ^the betwe'en-groups 

. - i " . , ' 

variance) and two smaller variates eacl^'' relating^ essentially to only one or 

two patterns of momentary interaction, , Because the transformed canonical 

variates are fairly highly correlated, it is clear that tl^ere is a 'second 

order genetal factor in operation h^re. It^is therefore necessary to take into 

account bo.th the 'transformed .loading^ (^eights) and correlations (projections) 

in order to interpret these facytorsJ The existencer of a second order general 

factor suggests that all of the transformed 'canbnical variates are getting at 

I ' ' ^- " . ^ 

different aspects of the same general distinction between a highly structured, 

directive, and controlie^i cla^sroop .climate and a more relaxed, free, and 

flexible climite (roughj^y/thef horiJzoatal ^axis in •Figure 3.20. ' 

/ 



Interpretation of/Diercriminant Function ^Ayes 

In an^ttj^'mpt to shed light upon the nature of the* four 'transformed • « 
' ' / /' '/ * *| ' •< . * ' ^ . » 

canonical discriminant function axes presented in T^ole 3»7* we scored every 

episode' 0f .classroom interactiorfj in thfe day-school sample, in eacn of these ^ 

• ' ■ • ■ - " , 

fout ^'ihy That is, we obtalnedf the appropriate|.y^ \feighted lineai^ombination , . 
of classroom ihteraction « factor /scores for eacb episode of interaction. We , 
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TABLE 3,7 



Jxja^lings .of Classroom Interaction Scores 
On Transformed Canonical Varlates ' 
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*then avferaged tnese scores' separately across all positive instanced of each^' 
category of ev^ry item in the original classroom' observation system. (Table 3,2)i 

.The resulting category means ate symbolically, depicted in^ Figures 3.^ through^ 

3.6, for th$ respective transformed d i^s or iminant function axes. There are 

ten colxnnns in each f igure, , corresponding to the ten major' sections^ of Tabic 3,?; 

the vertical position of each categoty label oorrespbjids' to its mean score 

(weiglit) on the :c)ntrast *ia question over all instances the beKaviora^ 

category\in' que trion. flacH figure tl^us depicts a contrast between classroom 

interaction patterns employed by teachers in "the day-school sample. The — 

method used here for weighting categories is reminiscent of the method of. * 

I • . ' ' • * ^ 

reciprocal averages as well as of multidimetisional analysis of contingencies 



/ 



, (Gutthian,. 1941.). , % 

lit must.be kept in Mnd that the four contrasts present?ed in Table 3*7 

and Figurjfes 3»31i^5j*3.6 weV-e arrived at because they discriminate quite sharply 

* j . ' ^ . «» * * . 

,betWeen behavior patterns observed in different day-school classrooms in this 

particular study. We have already seen that the classroom differences 

• • ' ' • •* * ^ 

detected are largely a function of teacher differences, however, so it. is 

\ \ "-^^ ^' / ' . ' • ^ 

quite likely, that' other contrasts in teachingx behavior would become salient 

«^ivenJ a different sajnple'of teachers. • 

First. axis. The coefficients presented In Table *3. 7 suggest that the 

i±x§t axis is a -specif ic dis.tinction between the 'Iquestionr^nswer-corrective* 

fe^dback-prompt-ansyex" paVadigm of classroom^nteraction and the "direct- 

,reVd" andyor "other" paradigm. >^ When we^get. back down to the level, of 



I" ■ • I 



Hie "time" entry in the first column refers to thpse episodes of interaction 
wh4^h*^endur^d for more than one minute,.!^* *• * . . . 

^If (fche reader, will refer back to Figure 3*2 it will .be seen that this contrast 
:caia.<be,apprbx*imatfed by tilting the horizontal axis about 30 degrees counter- 
^criockwise and then .reflecting the ;4ir^ctlo,n, of scoring. ' ^ ^ *' 
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Classroom Observation Category Means on Transformed Canonical Discriminant Axis I' 
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Classroom Observation Category Meaas on TrangformecJ Canonical*' Discriminant Axis I 
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Classroom Observation Category fleans on Transformed Canonical Discriminant Axis 
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contrasting individual aspects of classrotfta interactioji in Figure 3«3, 
however, we see that a' good deal of interchangeability characte^rizes the 

follow-up aspect of the positive pole of this contrast.^ We see that this 

>- , ' « 

'^question- answer" pattern directed to individual students and followed by 
"corrective feedback" may l^ad tp^quite a variety of teacher behaviors in 

addition to "prompt", "^qdent-student feedback" likewise figures prominently 

, - '4. 

as a follow-up pattern, as does "student quest i'dn^asl^ing", "Questioning" on ^ * 

the part of the te/^her is likely to involve "objects" or "rods" and 'a "free 

response" strategy may just as well prevail as a "question#*and answer" 

strategy.* ' . V ^ 

From Figure 3.3 we can also se'e that the "directr'read", "other" pole of . 

this first contrast between^ classroom Inte'ractiorl patterns is characterised 

by m^ny. subtleties at the level o£ individual episodes of- interaction. 

The likelihood of teacher "explanation", "writing on the board", andj "modelin^^ 

noteworthy, given, the* indication that a time-consuming strategy o^f 

•^repetition", "directed dialogue", or "discussion" is in effect. 

Second axis . From Table 3.7 and Figure 3.4 the reader can see that the 

second »transf ormed discriminant function axis picks up a contrast: between the 

"free response" mode of interaction and the "model-practice-corrective feedbaclc- 
♦ • 

model-practice" paradigm. As for ail ^our transformed' axes, 1:he positive 

pole^ applies more to indilidual:tzed instruction while the negative pole 
• * 

reifiers to class or group-oriented iristrjuction. It is notfewrathy that the 

^'free^' response" strategy lea3s to "other" categories of feedback and €ollow-up 

* » * 

behavior. ' 




1 ^ ^ ^ ^ * ' 7'^ ^ ^ J, 

'Refcrr.ing back to Figure 3.2 the second conti^ast can be visualized Uy--«.lting 

the horizontal axis .abput 45. degrees clockwise ^d reflecting the direction 

of scoring. ^. , * * • ^ 

". • ■ *: ' 111' ■ ' 
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Third axis . The third transformed canonical variate is not a strong t 
bipolar contrast, unlike the others, but relates quite simply to, the presence 
of "stfUdent-student feedback" as a component of classroom interaction. Alo/ig 
with this^ student feedback ^is some indication of corrective , feedback on the ^ 

" • . ' ■" -• . • ' " 

part pf the teacher and other follow-up behaviors similar to those seen for 

I - !■ ■ ' / • . • ■ ^ • • • 

the j^irst*-axis. • ' , . 

' & . .' ' * . 

Fourth axis. Finally,' the fourth transformed canonical variate is interesting 



i'n. tlj^t i^t "relates strongly and negatively to. the original bipolar contrast found 
77hen*we* factor analyzed classroom interaction patterns as they varied from 

— 5 " \ ' • ' . ^' 

episode t<> Episode: . "teacher, quest ion" -"student answer" vs. "teacher direct"- 

udent read and/or ask qCiestion". We can now reiterate our earlier conjecture 

khat, the "<|irect-read/ask question" pole of this, factor is, irt^eed, an inter- 

|ction pattern characteristic of -relatively free, unstructured classroajj^s, 

w^reas the^ "question-answer'^^p^leyzharacte^z^s ^moxe highly^ structured * 

cllssrooins.. "^his argument is\borne out to soii^e extent by the positive signs 

of hie correlations (albe:(,t low) between thi^^an^f ormed canonical variate 
\ >\ I ■ • . ' ' * 

and me otliers,\ taken together with the fact that the original QA vs. ^DIR . 

\ ' ' \ " r ' • * : * * * 

V score Vas a neg^ive weight ia Table 3,7. From Figure 3.6^>e see that "other" \ 

goes al^ng with ."direct" as the teacheV behavior characteristic of the^ positive , 

pole of ^he conti;a\t in question. Likewise,, it can be seea that sightwords 

or cartoo^ ar^ the\likely stimuli whi^di students are directed to reW and 

ask questions al^ut. On the qther pole of this contract between teaching 

/ strategies wfe now see emphasis upon the "model-practice" pattern of initiating 

interaction w\th th^ class. . * ; . . 
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What the ttansformed canonical discripihiirft function axes seem to reveal; 
is .a aettaitt ampunt of coinplexity or ^ultipLicity of determination on the par*^' 
of sevei^af ^attenjs* of manifest •clasairoom interaction. Thus, the observed 
interaction pattern "teacher 'direct^'-j-^student read" can either be part of a ^ 

^ . ^ 

♦ f , * . > . 

highly ,structure^ "programmed" cla^isroom interaction climate (the negative ; 

^ I ' «,* ♦ •• 

.pole of the fir^t axis) or .part of n^re open-ended, unstructurved setting . 
for interaction (the /positive pole of the fourth axis). Looking at the 
opposing poles, of these same axes, respectively^ We can infer ^hat "teacher 
asks question" ^can likewise be part of an encouraging, prompting, follow-pp 
pattern or it can forn!*part of a more demanding .pattern of drill, t 

. , ] ^ ■■ - ~ ■:. 

Of course^ we have, been^ attempting to identify the transformed dlscijiminant 
^^%ces with hypothetical constructs which migh9 ynderlie and 'help account for 

■ '■ X ■ ■ ' ■ - * 

y^r^tion in observed patterns of clafsroom^ interaction, Jn so doing we must 
^ somewhat, beyond the empirical results to^engage in conjecture and speculation 
the readei: is thus entitled to their own interpretation of the transformed 
disclriminant: axes or may abstain from theoretical speculation. In this 
connection we canViote that the particular orientation arrived -at -through^ 
transformation i^ completely arbitrary ih, the sense that it haS ho effect 
whatever ^on the total ,^mount of between-classroom vW^^ce accounted for* 
it is only when we begin to attribute fractions of tl>e total between-cl^ss^ooms * 

\ 1 ? ' • ..." - ' 

variance ito one axis or ahother that the orientatidn of transformed axes 
^%ust be fixed^hy -some means, (e.g., canoniisal form simple sttucture).. 



/ • \ ... 

RELIABILITY OF CLASSROOM" INTERACTION PATTERN .CONTRACTS , ^ 
^ The sijccession of data reduction procedures applied to clas^jroom behavioral 



o^jlrvations has ultimately led tq onl^/56 essential numbers, tHe classroom 
ceAtroid coordinates on each of four transformed discriminaht function axes. 

—r^ 7~--- ^ ■ -^^T-fT" 

These coordinates jief ine four ^contrasts between day-schooT cla.ssrooms whii^h 

. ' ■ \ ". . , '\ 'I 

can now be used* in an attempt to predict stud^ntf achievement over the school 
.year from classroom interaction experiences. Sow reliable are ^feese indices 

of classroom interaction* since unreliability limits. pi^edictive validity? THe 

' . . - ^ ' • ' ' J '''' 

issue of reliability also provides us an opportunity to return, as promise^^ 

K ' '* ^ ' * / 

to the univariate analysis of vatiancre results presented in 'liable 3^4',, sinci^ 
- " ^ • - • ^ ' / ' - 

we will discuss reliability in terms of variance component^. ^ ? • • v 

We stated earlier that the nine factors of momentary classroom' intei?a^tion 

' * ^ ' ' ' ' ^ ■> ' \ ' ' - * 

* "^detec'ted by anaiLyzing the total day-s(5hool sample of some 7,000 Episodes were 
" — -recovered in substantially the same form even after partialling out all tefacher 
and cl&ssroom effects. This result led us to conclude that th^ factors were 
not due tq any latge extent to dlfferencies between teachers. The ^analyses of 
variance smmnarl.z'ed in TableO.4 indio^te, however, that large -difference's 
exist between day-^school teachers almost every factor. ^ We can ^reconcile, 
these resultj^by considering components of variance in the total s|imple.of 



.classr:o6m eplsodes>-,.J]5r 4 ^' 



. Components of Variance Analysis of Classroom Interaction Fac|:ors 

The sample comprises a completely nested four-leve;L analysis of variance 
design haviijg approximately :^0 episodes nested within each session>^feproxi- 
WV;.;-,, ifi^.tely 25 sessions per classrool, and about two f lass rooms per teacbet. 



\ * From Table 3«8 it can be seen that from 53 to 90 percent of tlj^ total 

variance of each classroom interaction factor can be attributed to within- 

session variation (se6 line labeled Episode in Table 3.8); i;e*, to^ variation 

from interaction episode to interaction episode duririlg the same session (day) 

of observation. The factors are^ thus primarily factors of within-session 

variatioA. . * ' ♦ * 

In Table 3.8 we have indicated the nested analysis of variance F-test 

'probability levels associated with each -component of the total sample variance 

along 'With the assoc^.ated degrees of freedom. From these results, which are 

.more complete than the simple tests of classroom differences .relative to • 
\ • ^ ' . * . ■ ' ^ . ' . 

,!Bession-to-.session variation givet^ in Table 3.-4, maAy aspects variation 

\' 'k' . • ^ • • ■ ' 

ik classroom Interaction .patterns can be clarified.' It is clear, for instance 
that only v^ry small and generally nonsignificant contribution to variation 
in interaction patterns can be attributed to differential treatment of 
separate classrooms by the. same teacher (see line labeled Classroom in Table 
3.8)— -an outcome already • suggested by the compact clustering of classroom 
centroi^s for most teachers in Figure 3.2 of the multivariate analysis. ^ 

In this connection^ it is noteworthy, however, that "student-student 
feedbacky (SfBK) and "teacher question-student answer" vs. "teacher direct-' 
student rea'^/ask question" <QA VS DIR) are. the only two classroom interdctipft 
factors for which very small and insignificant teapher differences are 
.indicated in Table 3.8. Recall that these two factors are heavjLly weighted , 
in the last two transformed canonical discriminant axes (Tatle 3,. 7, Figures 3. 
and 3.6); We will shortly see that the latter two axes are ' somewhat sensitive 
to differential handling of separate classrooms by the same teacher. ■ Notice 
that the classroom interaction, factors ha^ve been' listed and partitioned in ». 



TABLE -3, 8 



Components* of Unit. Total Sample Variance of ^Day-Schodl Classroom 
Interaction Fac.tors Due 'to Teacher, Cl^ssrooia, Sesslorl, and Episode * Dlfferertces 



Sojurce 



Teacher 

•C],a88room 

^Session 

Episode^ ^ . 

, biscriminant 
Axis - 

. p< .05 

.01,. 



■QA tSQ CF-P-A OTHER DIR READ TREE MOD PRAC ' CF-M-P • , SFBK • QA VS.DIR. 



■ ** ~ ** • ^ * ^ 
IJ.O 9.4 1.8 5.9 



0.0 



.2 ; 0.0 



0.0 



** ■ ** ■ ** ** 
15.2 12..7 - 30.1 24.4 



68.8 77.6" - 68.1 69:7. 



+1 



+1 



-I 



** 



11.6- 
* 

•1.6 
33.9 
52.8 
+11 . 



4.e- 

,'0.0 



** 



11.0 



84.4 



** 



•4.6 



•** 

•1-.5 , 



** 

10.8 
83.1'. . 

-li 



.5 '■' 



.5 



** , . • 
1.8, . ■ . 1.0 



7.5 



90.2 



+III 



*33<8 , 



4 ' '>> 

5^ 
8 



368* 
6704 > » 
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Table 3.8 in 9 way^.whi^ reveals their condeoirion to t^he transformed canonical 
variates in TabUe 3.7 — the 'signed Roman tiumeral at. -the^ bottom of each column 
is to remind the reader of th^ major weight received by eaqh classroom inter- 



action factor ^ defining the transformed discripiiri^t funptibn axes. 

We can now reconcile the discovery of highly significant teacher ^ ^ 
differences on. factors which are primarily "indicatof s of episode.-to-episode 
variation within the same .session, iii any giV^n classrpom. The components of ^ • 
variance in Table 3,8 indicate that we jgairi little practical information* about 
what might take place in. any given'-3p£sode of cl'assi^om interaction through 
.knowledge of which classroom is being observqfd (from .5 to 16 p^ercent of 
variance can be accounted for^ line labeled Glassroopi iii Table 3.9) • In 
order to^ account for the variance ktapng episodes in a theoretically useful 
way^we are thus IrfEt to' the, postulat'iOn of hypotheti^cal detprmitiants 
(classroom interaction factpjs) which apply regardless of which teacher i^- 
involved. • * " . 



The relatively large components <>f variance for Sessions suggest* that a 
pumber of "these factors fluctuate* substantially from day-to-day. Especially • 
noteworthy are the large components of variance for interaction patterns 
involving "free response" (33.9 percent) /"other"'*^(30.1 pe*:eqLeht) and teacher 
"direction" (24.4 percent to 33.8 percent) ;beha^iors. Thes6 results suggest 
that any given teach^r:~might be inclined to ^devot^^j:ettaih sessions to these 
•^Specialized" activities but refrain from doing so in%ther se^lslons. Note, " 
however » that considerable seSsion-to-sesslon variance characterizes many of the 
classroom interaction faptors, .This tesult indicates that ^ach session of 



...... ^ . 



^Some of th^ session-to-session fluetiiation could b6 .attributed to observer 
biases, since various obsjervers collectedl data pn-^^dif f erent sessions. This^ 
suspicion/ will be dis^^elled shortly, . however, when wb jionsider the interns 
observer agreement apit>roach^to.reliabiri.t.2j^5 * 



observation provider a rather narrow view of what is going on in the clajssroom 
in genexal and niay explain. why stable teacher differences could no^t be well 
established with the limited night-school observation schedule. * 
♦ ^ In sujmnary, difference^ among teachers account ioT from^.5 to 1$ percent 

» of the totaX variance in the observational data (see line labeled Teacher 
in Table 3.8). '•The differences amprig^the teachers* ate statistically significant. 

^11 but tv/o factors (SFBK ajid QA VS, DIR). In o;rdtnary 'lan&u^ge th^ .teachers 
teach differently, ^d they differ rfost; on the factor "question-answer--^. 

^individual student" (QA IND) ^nd/'free "^sponse*^ (FREE). There i^ nat' muoh 
evidence that the particular classroom being handled by a teacher has'any 
Inflixence upon the patterns of* interaction which take place therei^, as we 



have, seen* earlier in connection with Figure 3«2« ^ The major component of 
overall variation in classroom interaction patterns^ can be attributed/to 
alternation in inter^tion pajttemg from episode^^to-j-.episode, which is not 
surprising, but session-to-session changes^ also play an important role.^ 

Notice that session differences are statistically significant on all iiihe 

i 

factors • *: _ * > . . ^ - 

Fortuj^iately, predicting what might take place in each individual 
classroom episode or even in any given session of observatibji was not our 
task in this study, so we can be content to account for such variation by 
reference to hypothetical determinants* or "factors" of momentary classroom. 
^Interaction. " Our aim was to detect stably classroom or teacher diff^erences 
and it is to aSsess our attainment of that goal that a reassessment' of the 
components *bf variance in Table 3*8. is required* That significant teacher 
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differences were obtained 4espi,te the fact that much' session-to-session and 
episode-to-episode variation occurred is not "tiard to understand because So 
many sessions and episodes were recorded for each teacher; > ' » ^\/' 

J^hen the components of variance in. Table 3. 8* are weighted to take into 

'• ■'.■'.¥'. • ' ' 

account the fact' that agjproximately 1000 episodes of 'interaction were, observed 
. per day-schopl teactjer, about "50p per classroom, and about 20 per session, 
we seq -that v^ry ^arge' fraction of the observed variation among teacher* 
means on each classroom interaction f actt>r is reliable (i.e., can be sittributed 



to true differences between teachers). Since substantially the same issue 
is addressed by the F-tests among teachers (given in Table 3«4y, it is cleaT 
why so many significant results were found in the 4ay-school sample. 

^ , Components of Variance Analysis of - * #i 

Transformed Discriminant Function Scores 

* ^ -A 

. : ^ ^ , 

' s t ' 

The multivariate analysis of variance (canonical discriminant function 
analysis) in Table 3.6 revestled only a few significant contrasts among day- 
school Classrooms* From th*e first four of these canonical discriminant ^ . , 
vfunctions j^e derived the four transformed axes discussed earlier in this 
chapter* ^ Since these are:,the''classifoom interaction contrasts which we' wish . 
to ejnploy tov-predict studente a^chievement, it is in order to determine how' 
sensitive teacher scores^on these tiontrasts are. to various possible underlying 
sources of variation: teacher differences,* classroom differences, session 
differences, "^^^sode differences, and observer differeax^s as well. The 
issu6 of 'intei>-observer differences will be taken up. shortly. 

, First, J;iowev^r,* let us look at nested analysi^s of variauice results, 
(components of variance) fo^r. the' transformed discriminant axes, just as was 



done for the nine original factprs of "-classroom interaction it> Tabl^ 3.-8, The: 
Upper s^ctipn of Table* 3» 9 (gives components of unit total sample variance fqr 
the transfoimed canonical discriminant function axes .comparable, to the entries 
in Table 3#8l From these entries we can see that there is substantial ^episode- 
to-episode* variation in all four of- the classroom interactioJ contrasts (from 

58 to 86 parcWt .of the total. variance)^ Note, however,^ that the maximization ^ 

11 ^ . ^ «^ ' . 

of be twe ear classrooms variance relative to session-to-session variation 



(defining, 
axe^) has. 



the 



upper section 



canonical orientation of*thje original! discriminant function 



datlried over from the analysis preqented-in Table 3»S in the ^orm 



of firge totai^oients of total sample variance attributable ,to teachers (Table 3.9 
Teachlsr dlf ^ereHces account for approximately one-fourth of the total ' 



I) 



observed variance on each of the first two ttansformed axes (I and II). What 
t;his means, ^ in a pjractical 'sense, is. that we could reduce our ' uncertainty , 
about which pole of either contrast might manifeist itself in any given ' 

4 ♦ 

episode of classroom interaction by a substantial amount merely ^by knowing 
which teacher Is in charge.* If tTie teacher is *high on. the first contxrast 
(tea'cher B in Figure 3.2), wfe would do well to predict ''question-answer- 
corrective feedbac]k-prompt-answer" interaction episode^! On the other hand,, 
if the teacher is low on the first contrast (teacher *F or •perhaps E) , then 
we would do we3;l to predict "direct read" and "other" episodes. 

« Con^dering the second*-contrast we would proceed in a simiJLar fashion, 
prediot;ing the "free response" mo^e of individualiised. instruction for the 
p<Jsitive pole (teachers D, B, and E), and group "model-practice-corrective 
fe^dback-^del-practice" chains df episodes for the negative pole teacher F). 

Even the last two transformed canonical variates ("student-student « 

"» _ ^ « 

'feedback" and "direct read. and/or ask question vs. questionr-answet,"/ • 



. . TABLE 3;9 ; -. ^ ' . 

% • ♦ . . . 

Component s^f Ujiit Total S.ample Variance of Transfomed Discriminant 
AKfes Due "to Teacher, Classroom, Session, atid Eplsode*Dlfferences 

¥ • " . ?P ' "ii m ' IV ' ' • ^ ■ 



ource 



Teacher . 22.9 25.7 ' / 4.5. 4*0" 

Classroom .8 .3 2.-1 . 1.0 



Session V ... 13.4 . 15.^8 . ' 7.1/ ^ . 15^.t . 



fiplsode \' . 63^0:.' ^ 58.2 '86.2 79.9 



0 



Relative Contributiqns of 'True Scqre, Dltferenc.es Between Tieachers,"^ 
^* Classrooms, 'Sessions, and Episodes to Observed Variance Am<mg ) . „ ' 
Teacher Means on Transformed IJi^SilnitinSrit A:i5es 



" ft. ' 

I ti III. IV 



-Source 

Classroom. ' LS/f . .5%. 16.5% 9.2% 



Teacher. ^ 97.2% 98.2% 79.'B% 0 " 8^.4% t. 



(. 



/Session . " 1.1% ; l.'l% 2.3% . .5.9% 



Eptsodf • ^ .2%*- .■.2% 1.4% 1.-5% 



respectively)* show larger teacher and classfoom <coTnponents ofrvariance in 
Table 3.9, than do the original factors' of classroom interaction .from which 
they ate largely derived, .However, it is clear that thfe latter are not 
major contrasts in teaching styles so we could**nbt use knowledge of average 
teacher performance on these variates to predict mu^ abou^ individual 
episodes of classroom interaction,; t^t us turn, therefore, to a consideration 
of how reliably the teacher means on these contrasts can be estimated per se . 

The lower section of Table 3.9 gives a breakdown of the weighted 
contribution qf each factor in the nested observation design to observed 
variation in teacher means on e^ch transformed cfiscriminant function axis. 

0 I 

Whereas the first twb contrasts are almost pe?:fectXy reliable indicators 
of teacher differences (97 to 98 percent of 'the observed variation in 
teacher means can be attributed to true differences among teachers), the 
last two contrasts are somewhat sensitiye to differential interaction in 
various classrooms taught by the same teacher. > * - 

It is clear that observation of 'more different classrooms would be 

■ ^ .^-^ 

required before we, could get a v^ry reliable indication of how inclined any 

given teacher is to allow '''student-student fe^back" (III) to tafcie place. 

From . an inspection of Figure" 3.3, moreover, it can be seerl that the original 

classroom interaction factor^ of "SFBK" is oriented in a direction Which .ia. 

« 

sensitive to variation in the way in which t€:achers C, D, and B handle classes 
which vary in level; more" "student-student feedback" tends to occur .in lower 
level classes. .Note the high episode-to-episode variation in "student-studlfent 
feedback" (III, opposite^ JEpisode ) , however, as 'wej.1 as the low session-to- 
session variation (III,* opposite Session) — tKls difference is diagnostic of 



tW ^generally low frequency with'^'which this form of interaction occurs (see 
Table 3^2) r * ^ . 4 ' ; 



Both more classrooijis. per teacher and more observation sessions per • ' ^ 

♦ ' 

classroom would be required to pin down precisely a^y given teacher's 
tendency to "direct" individual students while they' "read and/or ask 
•.questions" vs* their tendency , to engage the entire class in "practice" 
sessions' With "mo'deling," "writing on the board/' "explanation," and ^ 
".questioning" (IV in Table 3,9). Apparently individual teacher^ are some- 
whit flexible aboi\t alternating between these strategics from class-to-class 
as well as from session-to-session, which helps to explain why a bipolar 
factor ^etnerge a in the analysis even when individual classroom interaction 
epi'p^des are analyzed. ' * /- ^ 

' The components of variance discussed above r.eveal that we dre^jiealing 
with highly reliable Indices of teacher variation ; the reliabiiytties of 
teacher me^ns on transformed discriminant axes r^ange from •80 to\98 in 
Table 3.9# But a more, important feature of lihis investigation is the evidence 
it provides that ^ those, classroom interaction contrast;s which show differential 
treat^to6nt of different classrooms by the same teacher are diagno^tj[,c of variation 
in the ability levels of the classes involved. Whereas our initial impression 

of vFigure 3^ holds true — teachers do have distinct and consistent styles o^f 

* ' ' ^\ 
interaction with- studen<^ as indicated tfy transformed axes I and II — there 

^ W ^t * i . " 

is also evidence that certain more limited aspects of classroom interaction 
vary from classroom-to-classroom taught by the same teacher . That the latter 
variation might be in response to student characteristics is^^of great interest 
4nd t^ll be taken up later. - / , - • 

Inter-Observer Reliability *• , 

A final question about reliability remains before we .can proceed to our 
inain* taslc o£ relating student achievement to teacher performance: do different 



obaervers record th6 same aspects of 'variation in classroom interacAon 



patterStrs? In order to study -the ^ssjae of po§sible observer bias i^"" data 

was/collected in some of the early observation sessions by paird of 

independent observers observing the same sessions. Since three observers 

were used in the study; it was arranged to have concurrent observation? - 

'made by each of the three pairs in eight different^ classrooms. ^ 

We are not in a position to make a strict comparison of inter-observer 
agreement in the usual sense, since observers work^ independently and there 
is no way to Imow which of their datsa-sheet ''entries should correspond. * 
(The relevance of this correspondence is frequently *^^nored in estimating'* 
inter-observer reliabilities.) However, we can compare data on the basis of 
individual sessions.. In Table 3.10 we have .summarised the results of this 
comparison for each of the ^three pairs of observers, using as the basic units 
of analysis session means on the four' trarsformed classroom interaction* contrasts • 



It is appropriate^ to loork at possible observer influences on these scores because 
we have used these scores, to predict differences iii student achievement. • 
Note . that we are discussing observer assessments of session-tp-session . , 

variation iri'theser interaction patterns, but ye have already seen from 

» ' ' ' A * .'* • " , 

Ta§le 3*9 that enough sessions of observation were obtained in the da^-school 

• ' ' * -i - 

sample yield highly reliable teacher means despite amy session-to-session ~ 

' I • . • - • 

variation. Since each classroom was visited by all observers; th^ influence^ 
»of aify possible^ observer "Triases on clasgroom mean scores is indicated -by * 
bession-'to-session ^variation , which as we have seen^, accounts for less of 
the variance than -teaqher differences (lower section of Table 3.9). , 

» From Table 3.10 ^Jt can be seen that session-to-session correlations 
among transf briaed discriifiinant function scores* derived from the datk ^ . « 
collected«concurrentiy*By different observers ran^' fro|a ^9^ to .SH for 
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^ 1 TABLE 3.10 



Means, Standard Devlatlonsi and Correlat ions for'' 
Pairs of Observers on Tran8*formed Discriminant Function Scores 



Yarlate Observer Pairs 

, 1 : 2 I* 3 



8 

r 



r 



X* '■ , • .93 ..70 ..13 .32 -"^ ' .33 ..52 



,8 ■ 2.36 r.89 1.62 1.47 ' 1.45 1.58 

t ■ ■ ■ -:98: ■ .99 ' .99 



-II X .65 *54 .00 .04 ' .13 .32 



■1.01 3^.13 - 1.39 1^73* 1.1"'7 .1.5? 

.-95 ■ ■ .■^e .98 



III X .47 ..08- .OS" ...59 ' .26 .47 

~¥ 1.42 '".96 • 1.82 1.82 1.54' 1.03 



.66 .81 ) .37 



IV , X ^ .87 - .'55- "-.13 -.09 - " » - ..11 .23. 

s ".2.44 2^29 1.50 1-.65. 1.83 1.63^: 

^ V ' ' .'97 i ■ .97 fX .97/ 




X " mean, ^ = standard deviation, =^ Product moment correlation; 
eight ol3servatlon8. . • , , 



126 



, \J ' ^.?«? • ' 
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all but the third axis.. For t^e-rlatter, the correlations suggest that 
^ certain observers may have had difficulty either recognizing or recording 
"student-student feedback in Spanish." Since the lowest c&r relations 
inrolve Qbserv^r "^2, it i^ possible tha^t this one individual, is the problem;^ 




however, none of these correlations are as high as for the other axes. 
One problem is that this variable (as we mentioned earlier) occurs 

^ 

relatively Infrequently — particularly in. the more advanced classes.' For 
this reason* each such occurrence is highly weighted (as seen in Figurfe 3.5j ' 
and the failure to record any given instance could "have a sizeable influence 
upon the outcome for that session. ^ 

When correlations comparable to those given in , Table «3. 10 are studied^, 
for all nine of the cfrigirial clas'sroom interaction factors, a similar 
conclusion is reached; only for "stuflent-stiident feedback" is there a serious 
problem with^inter-bbserver correlation. These results a^re briefly summarized 

in Table 3.11. ^ [ ... 

* -^^ ■ . • 

Finally,^ it can be pointed out that inter-*observer correlations for 
different scores were much lower than for the same score , Indicating that the 
observers were indeed discriminating between different classroom interaction * 
patterns. . ' ' > . " . ^ , 

* - " . *» ^ 

NIGHT-SCHOOL TEACHER DIFFERENCES ON CLASSROOM INTERACTION FACTORS 

. As we saw in Table 3.4, fewer differences were detected among night'^school 

. \ ' ' ■ '/ ■ -^'\ ■ ■' 

classrooms (only one classroom per teacher) than amcmg day-schcx)l classrooms. 
Thi's\ result could be due to a real lack of differentiation, of t&^hing styles 
in the night-school, or to the low number of sessions of observervaK^ons per 
teacheV> 



^Notice \from Table 3.1 that observer 2 collected very little data in. the study.. 



TABLE 3.11 



Interobserver Correlations for bfine 
Classroom lataractioa Factors (8 Observ^tio 



-Variate 



' 1 



Observer ^airs 



X:FrP-A 

pik^READ 

MOD. PR^C ' 

OTHEl^ 

QA vs ; DIR 

QA-IND 

CF-M-P 

SFBK • 

FREE 



,9r 

.97 
,96 
,79 
,98 
,99 
,82 
,13 
,94 



.99 ' 

.93 

.98 

.99 

.94^ 

.98 

.96 

.42 

.96" 
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X analyses of the classroom interaction fa'ctofs for the night school 
paralleled' wha.t was done for the day schoo^.. We will only present a brief 
discussion of the night-school results in what follows, however, because of 
the limited-data, available. As for the day school, it is in order first to 
consider day-t6--day as sociations among classroom interaction factors (pooLed 
within-classrooms) , then to move on- to classroom (teacher) differences^ 

Table 3.12 gi^^^*«*^-(pooled) correlations among night-school classroom 
interaction factors as they vary across sessions taught by a]jy given teacher.* 
These Gorrelatioas are comparable to those given in Table 3.5, and w^ have 
again orde^ed^th^ variables so- as to reveal clustering. 

^ J' 

Hie reader will notice that session-to-session covariation of classroom ^ 
interactiSn factors is ^uite different for the night-school and day-school [ 
samples, a contrast with the high degree of similarity found when we^ ^ 
compared; structurai'^atures of the individual classroom interaction factors' 
for the two samples. It suggests that the same factors of classroom interaction' 
^ are present in both samples at the l6vel of individual episodes, but that 
-v^iat^pn in overall classroom organization is different in thesii^samples* 
The night-school correl-^tions show a tendency for drill' ("^irect--read"; 
"model-practice-model-practice") to vary in conjunction with prompting 

i • Y • ^ 

Jt"t:orrective feedbackrprorapt-answer"; "student-student feedback') across 
sessions. By and large, these associations are not as easy to interpret as 
for the day school. 

• r * 

The first two canonical- discrirairiafit axes account for 73 percent of the 

variation between the six' night-school classrooms (relative to pooled within - 

^ . / - 

group variation), as can be seen from Jatle 3.13. 
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TABLE 3.12 

Pooled Within Night-School Classrooms Cprrelations -Among Factor Score Session Means 



DIR READ 


HOD PRAC 


SFBK 


CF-P-A- 


bxHER 




• QA„ 


QA IND 


QA vs. DIR 


-> 


1.00 




















..69 


" 1.00 


> 


■ 

- 














/•■•" ,• 


'.32 


1.00 


















.23 


.28 


1.00 










• 






- .16 . 


- ..09 


'.62 


1.00 


1 










I- .46 . 




.06 


- .18 


-' . 02 


I'.OO 






a 


1 

to 


- .13 


: - .33 




.31 


.10 


-.08 


-1.00^'^ 




1 


- .11 


.14 


- .38. 


- .39' 


- .30 


.07 


- .04 \ 


. 1.00 ' 






.12; 


•0^ 


- - .35 ■ 


-/'.08 


- .13. 


' - ^28 


.17 


.24 


1.00 


« 



Mnemonic label 



Integraction pattern 



DIR READ 
MOD PRAQ, 
SFBK . 
CF-P-A 
OTHER 
CF-M-P 



/^QA IND 
QA vs. DIR 



• .s%«,^^treacher. direct." r "student read" 

. . "teacher model" - ''student practice" - "teadher model" - "stuji'en^ practice" 
./ . "student - studeiit* feeHbaek"^ ^ ' *v . ' 

! "corrective feedback" - "teachet prompt" "student answer" » 



"othef" 



^ . "corrective feedback" - "teacher model" - "student practicfe" 
. . "teacher question" - "student ahswet:"' (follow-up) 

* . . "teacher^ questiori -'individual"-- "student answer" ^, 
. . . "teacher.ask question'* - "student answer" vs. "teacher direct" - 
* "student read and/ or ask question" 



I3i 



TABLE 3.13 



NigHt-Sphool Classroom Observation Data 
Canon;Lca>jL Discriiainant Functioi! Anal^^sts 



Discriminant Function 

1 ' 

2 ' 

" .1 .3 
4 
5 



. Root 

'X.4645 
■.7973 
-.5945 
■ •.17.96 
- .0662 



Wllk's A » .112505 for F (45., 164) = 2.30 



Cumulative %• Trace 



• 47,2 . 

• *73.0* 
92.:^ 
97.9 . 

100.6 

:r 



V 

\ 



. Te'st for .residual after removing: 

0. First dlscrlpilnant function " . 
First two discriminant f-unctions 
First three dlsorlmlnant ^functions 
First four dlscrlijilnaht functions 
' First five discriminant furtctions 



2 . 
" 2L. • 

92.8 
54.5 
29.6 

2.7 



df 

45 
32 

-,12' 
5 



Variable 

DIR READ 

MOD PRAC , 

SFBK 

CF-P-A 

OTHER 

CF-M-P 

QA IND' 
DIR vs. QA 



Dlscrlmlnajit Function. Weights' Scale'd ^ot Unit 
Wlthln-Grpups VarJLance-on Original Scares f 



Discriminant Function AxeS" 



.029 
.027 
.095 
-.219 
-.060 
-.051 
.J 175 

-.134 
-.018 



.II 

-.217 
:-210 

-.'012 

. .191 
.075' 

-7054 
J.149 

fc', 095 
-.lis 



-fix 

» 

.0261 
-.034 

.22^ 
-.008 

.152 
-v040 

.087 

.269 
-.020 



■ r 



V..- 
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Just as for the day-school data, we have plotted classtboia centroids pn " 
the first two canonical- discriminant function axps in Figure 3^7 • Night-schoQl 
teachers have been identified ^''^^jjj^ figure wi^h 'the letters G^through L and 
we have also included vectors ^representing the original nine classroom inter- 



action factors. .Interpretation of thi^f figure can proceed In the same manner 
as for Figure 3^.2^ Again, we find the night-school results to be less * 
intuitively compelling than thfe day school results. 



SELATION OF TEACIplR BACKGROUND CHARACTERISTICS TO TEACHING STYLES 



As mentioned at the conclusion of Chapter, Two, considerable information 
was^^btained about each teacher in both the night and day-sfehoo^ samples 
through a questionnaire administered at^he 'outset of the study* It is of 
interes^'to see whether a teacher^ s performance in the classroom might be , 
related to this prior information about his or her education, experience, and 
preference among teaching styles. ■ . % 

^ Only 12. teachers wer,e studied, background characteristics* were often ^ 

badly skewed (e.g., number of years of teaching in TablB 2.7), and information 
was available -for more background variables than teachers. We therefore 
decided to use rather crude data reduction techniques in order to break these 
^J|a^ down to a inlnimal set of contrasts among teachers. .To this end, we 
rank^rdefed the 1^ teachers oxi each the firstly variables in Table 2^7,7^ 
then we. computed a matrix of squared di^nce^ among all pairs of teachers 
by summing their squared rank-dif f erence$- across all 17 variables. ' We then ^ 
factored the matrix of scalar products ^ound.by taking - 1/2 the.doubJ,e 
centered matrix of squared euclidean distances among teacher^ (Torgerson^ 1958), 



:i: eric . ■ . ^^^^ 



The ,f irst two eigenvalues x>f the scalar puroducts matrix araong teachers" ' 
. / * ' . ^ ^, ^ ' ' \ 

Were very prominent', suggesting a dimensionality' of twd^, teacher scores* on 

the first tWb principal axes \ifere correlated'' with the raak-ordered variables 

^and the resulting matrix was Rotated* to ofthagonal simple structure. The 

• • ■% , ^ • 

outcome of this "crude approach'to metric multidiinensional scaling, is 'presented . 

in Figure 3.8.' In this figure teachers are represented by the same i'etters 

used in earlier figures (A-F for day school, G-L fo^ night *schoolj)^and their 

background^^haracteristics are represented as .-vectors. 

Despite the approxiinarte nature of the analyses under discussion, 

I^ure 3.8 reveals interesting inf epilation about the teachers included in tHis 

St'jidy." The first thing to note is that^^he horizontal axis ib aliened Vith the 

variable "peffeent audio-l/ingual" and* it marks a xather strdng contrast between. 

day-school and tiight-sch9ol teachers. Only one day-school teacher, J, is placed- 

Coward the ''silefit wa^v (left) pole of the hori-zontal, axis. ^A.s-'for the vertical 

axis; it is cl^^rly an Indication ^of edjicational level .which differentiates 



teachers within both the day/Snd ni'ght-school samples^ It is n^^teworth]^ that 
"years .of teaching" is closely . aligned with this vertfe:ai axis but ^at ^i^uaf 
of experience teaching English as, a sej^ond language to adults ("ESL" day aduft") 
is -aligned vith the "silent *7ay" pole pf fhe horizoi:ital axis. In other word$, 
the more*experience a teacfief * has had, teachlifg Engfish as a second language ^ * 
tQ adults, the. more likely he or she is/ to iTse^f appYoxiii^te "the silent waj^»!^\ 
The more experience a teacher has h^d in teachingAn the^^em^ntafy or^ secon^.dify 
schools, the mpre likely they*^ are to use the audio-lingual method or an approxi- 
matiort to it. ' . - , - ' * . 
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Figure 3.8_ \ 



^ , . Multidimensional Scaling of ESI, TeacITers <5n ^ W 

tJhe Basis of Similarity in Their* Background Characteristics; ' 
/ O . Day Schbol Teachers (A-F), Night School Teachers (G-L). 



•It seems that the lower the level (from adult to child) at which an individual 

has had experience teaching, the more likely 'th'ey are to be educated, experienced, 

• * ♦ * 

.and a practitioner of the audiblingual method (e.g., teachers K ana L). 

' Comparing Figure 3.8 to Figure 3.2, there appears to be some association 
betyeen classroom interaction patterns (teacher performance, if you "will) and 
teacher background characteristics. Notice that the day-school teachers fall 
into roughly comparable circular patterns (from *A through F) in Figur^es 3.8 and 
3.2. As usual for tUe night school, however ,^ the comparison of Figures 3.8 
and 3.7 proves confusing. • - ^ 

- • ./ • 

An objective way to T:ompare the subjectively similar placement of day-school 
teaches with respect -to theii; teaching behaviors (Figure 3.2) and theUr back- • 
ground characteristics (Figure 3.8) has recently been brought to light by Schultz 
and Hubert (1976). It is possible to compute, a quaixtitative measure of agreement 
between, the two .sets of day-school inter-t^acher distances represented by 
Figures 3.2 and 3.8; a kind of Coefficient of proportionality called r in 

• /V- 

the literature or quadratic assignment where it was developer!. Once this 
coefficient is available it is possible to assess its probability on the tasis 
of random reorderings X^.g., mislabelings) of the teaciiers in either figure. 
If the correct labeling gives an improbably high index of .agreement between 
the two ways of getting ^t similarity , among teachers, "then we have objective 
evidence that the configurations conform to one anoth'fer^f Based upon the 

,d±staaces between teacher centrpids in Figure 3.2 (combining classrooms tadght 
by the samie teacher) and in Figure 3.8, we get T = 566.5. Sinc6 the meau F 
coefficient from all possible permutations .of teacher labels iii one figure 

.while holding the other fixed is F = 49.6.2, with a standard deviation of 36.3, , 
the degree of agreemeiit shown with the correct labeling seems unlikely to have 
occurred by chance (p < .21 from Cantelli bound which is conservative).^ 
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[ . CHAPTER FOUR 

the' .relation of teaching PERFORl^CE TO STUDENT LEARNING 
* « • * " 
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\ ' > • ■ - . - ■ 

' STRATEGY FOR ANALYSIS OF STUDENT. ACHIEVEMENT 

* \ " » ' 

The major proli^em to be solved in the analysis of the student achievement 
data was to determine how we, courd account for variation in posttest scores. 
There are three domains of variables which are potential predictors of posttest 
scores^: background characteristics of the students, students' initial level 
of proficiency as measured by- the pretests, and classroom experiences. 

Variables describing the students' backgrounds are worth considering as 
potential predictor^ of final achievement because these measures may be indirect 
indicators, of aptitude for learning, of academic skill, or of prior achievement 
of proficiency in Eng'14;Slx. Obviously, the students' initial proficiency (the 
second domain mentioned above) may be associated with their final achievement 
status. Pre- and postyinstructlon scores on the same test are usually highly 
correlated with each other because the experience acquired in the interval . 
which separates' these measures does not, greatly alter the relative ordering of 
Students' with respect to their abilities; the Jatter have,' of course., been 
built up over an entire lifetime of ^experiences, for which the* background 
measures are indicators or proxies. Nevertheless, the relationship between 
pr^- and posttest scores can be altered somewhat through the influence. of 
intervening ev^ts, includAg classroom interaction experiences. This is our 
third domain of predictor var4.ables and is t)f most interest in this s'tudyj - 
because it is ^the only domain over- which some degree of control can be , 
exercised. ^ ^ ^ ' * . 

Other relevant experiences which might intervene between^re- aiid postteS^ting 
'(such as use of English at home, on the job, and in the community) unfortunately 
could not ,be objectively measured in this study. The possibllit^r of controlling 
these exti^acurricular sources of experience is slight, in/any case, but it 



\ 




would be useful from a theoretical point of view to^take them into acQount. 
The best we can dp*'at this point/ however, is to bear in mind that certain 

* "baclcground" characteristics might serye asT proxies for^ sustained extra- 
curricular experiences, (fe.g. , occupational level far the ne^ to speak 

: English in the workplace; .length of tim'fe in the United States for assimilation 
^ into an ethnic community) . ' . . : 

' Given Ihese' three domains of predictor variables, which are organized 
" * in an .obvious temporal sequence (background experiences — > pretest 
. perfqrmance — > classroom 'instniction), the' analytic problem is to find 
.how the 'incarnation can be most parsimoniously combined to predict final 
achiev.ement- Multiple linear regression can be applied to this task, since 
the squared multiple correlation between a set of predictors and a criterion 
variable (e.g., pQSttest score) indicates what proportion of thS observed 
' variance in the criterion can be accounted for ^by prior knowledge about the* 
given set of predictors.* ^ 

,In this case we have three successive sets of predictors which, can be 

4 ^ . ■ ' ' ' * 

•taken into account in a logical sequence in order to attribute as much 

- influence as possible to .those experiences which have, priority in time 

. before entertaining more complex (and yore recent) determination of final 
« 

achievement. Background characteristics must be taken into account first 
-2 because these variables, are; direct measures of or proxies, for educational 

. attainment, competence t© cope, with the proce^es of . schooling, motivation ^ 
' N and- aptitude. If .such variables account "for most of the variance in final 
scores, then' classroom^experieqces can have little differential influence on 
final status; this is not to pay that large pre-*, to ppsttest gains could not 

• have -occurred, only that they are not likely .to have changed the relative 
ordering of student^ in terms of achievement. 
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Next we mus.t consider how much Initial status In terms of pretest scores 
adds* to the prediction A>f outcomes beyond what we have been able to learn 
from a knowledge of background cha^ifecteristics alone, A related issue here, 
is ho\L,well initial status per se can be predicted from background character- 
istics. This a^ialysis tells us how adequate and useful our information about / 
backjground experiences is in the first .place, . Had we in our' possession all 
of the backgr^^i^ irtformation truly relevant to achievement, it is clear 
Ch^t we would not have to measure initial prof!hcisi^cy at all. Enough background 
material would be available to predict pretest score^yaccurately within the ; 
limits of their own' unreliabilities. Of course, this ideal state of affairs ^ 

caniiot generally T>e approached in practice because of limited sampling of 

• < ' . ' , ' ' \ 

background Characteristics, measurement errors,* nonlinearities, etc, 

. The final stepr In the regression analysis Is to assess the unique contribution 

of classroom experienc'es in the prediction of f ina^Ts^chi^^Tfejnent, above apd beyond 

any predictive utility. of in^ial status and background experiences,^ We also \ \ . 

must determine , whether different forms of classroom interaction appear ^o have. . ' , 

different effects upon achievement. The purpose ;,of relating classroom inter- * 

action to achievement is to find out how much this information adds to t;Jie 

prediction of achievemerit and to identify any components of classrpoDS inter- ' 

action, which can be hypothesized to have aa impact Hipon specific forms of " " 

' ' ' •■ • ". '. ■ . ..''.^ " 

' " achievement . i ' ' * - \ ^ ! ; 

In sum, the strategy is first to see how well student achievement ' can 

be predicted by the, background characteristics of tlie students, then to ^ • 

" ^\ ' . . c ' 

.*/iet ermine how .much the prediction of posttest achievement* can be improved by 

addiiig information .about, pretest performance on all of the fall achievement 
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measures* The reader ^±11 see that both of these se^s of^ variables do predict 
final student achievement, as might 'be expected.' 'The .essence of the 



an^fiysis , 



however. Is to find out how. much of the variance In final student pex6<yrmance 



^chl€ 



can be predicted f rota classroom experiences* once prlor^ experience and .achlevey. 
ment are taken Into accojunt. . 

. * IJotlce that the classro'om experience variables differ fundamentally^ from: 
all of the other varlal^es In t^fe^nalysls-ln that they * are not individual 
measures,^ , but ^apply to all Individuals In each classroom equally. 'From the 
point of view of this study, th^se variables are the "Independent" variables 
whose possible effect upon final achievement we are most Interested %n 
learning. These^ variables take the form ofS:ontrasts among classrooms In 
terms of observed patterns of teacher-student Interactloji, the four transformed 
canonical varlates^ developed ar\d discussed Ixi Chapter Three. Every effort 

r ' * \ 

was made to arrive at a small set of Independent variables which are reliable^ 
and n6t tyoo highly Intercorrelated, in line with tlie .requirements 6f the 
multiple regression model. . On the other hand, the bactcground and pretest 
variables are more error pronfe; but they only play the role' of covariates to 
a^jusl: for preexisting differences among classrooms ,in the final analysis. 
Moreover, these data are avall'able for every individual, so stable estimates 
of the required regression parameters can be obtained .even with fallible ' 
measures. . ^ ^ " , - * 



' ' 'Predicting Fall and Spring Student * ^ 
' Achievement from Stud'ent Background characteristics 



.♦'^ It is conceivable that student achievement 'can be accurately predicted ^ 
from knowledge only of certain characteristics associated with students' % 



backgrounds. These characteristics are proxies for. previous educational 
achievement^ for sociocultural conditions which promote Mucational atta 



ttainment. 



and for motivational factors that contribute to greater educational achievement • * 
It. is possi^la that such factors account almost totariy.for differences iij 

acfttevement because the students in effect become their own teachers and are « 

• A, 

not highly dependent upon the specific teaching performances of the teachers 

■ ' . . . ' ■ . ' \ ^ 

Jto^ whom they are exposed. ' . \ • ^ . 

^ In Table 2.5 we saw many significant zero-order correlations ^tweem . . 
student characteristics and student a^ievement scores, both in the fall 

» X 

•and in the spring. The background ^and achievement domains are clearly 
related at the lev^l of individual pairs of variables, so— it is in order to 
se6 hOwJbackground information can be optimally combined (weighted) to predict 
individual achievement scores and to determine how effective this prediction ; 
can be When simultaneous use is made of all available, background information. \ 

^ Each achievement^ test score was predicted from the background -characteristics > 
using multiple JLinear regression. The.'^sults of these aftaly^es for the ^ay- . ; 
school. saiople are summarized in Table 4.1. In that stable the labels, across, 
the tops of the columns, designate the fall and spying achievement scores;, note 
thdt Itgarithraic transformations Of the 'oral proficiency scores yere predicted. 



The rows of the taf>le list particular background characteristics used in the "^^^ 



1 ' * ^ , ' 

All of the Oral Pro^f^ciency Test scores, were subjected tq_a logarithmic 
transformation before being entered in the regression analyses. The ^ 
logarithmic transforraation^wSTs found to be necessary^ because the original. 
Proficiency Test score variances within different dlassfooms'^were not' 
homogeneous.; i.e., the variances were highly coxielated wjfth clas'sfbom 
means. What this suggests i^ that, ^^ratfier than responding in an additi'v^ . 
fashion ^o\any factors which might differentiate classrooms, pral pjroficiency 
is in a sentee multiplied by such effects. Not only does the iogar^hmic 
transformation stabilize varianc^es, it J|ticreases the linear predicjtability 
of proficiency scores from background, pretest, and classroom interaction 
variables. ' - . ♦ . • ^ 



2 ' \ • . 

regression analyses. The niunblr in aity cell of this matrix is^ the 

standardized regression weight for a particular background variable (rpw) 

as a predictor of a given achlevenent score (column). These entries comple- 



mjpnt the zero-order correlations already given in Table 2.5. The f-prraer show 
the direct contxibutio^n of variation iu each^ background characteristic to 
achievement, whereas the enttifes in Table 2.5 do not take lato account, the 



fact that the background characteristics are correlated jmong themselves 
(i>e., they are redundant). . * < • 

The line in. Table 4\1 labeled ^ives- the squared multipl^^rrelation 
of each aqhievement test , score with ^ll ^of the background variables, taken 
simultaneous lar. These^%umbers indicate the proportion 6f the variance in 
each achievement score (predicted by the jgpmpllte set of background character*- 



istic variabj.es; for 'example,' tMe background characteristic variables account 
focJ26 percent .-(..26) of the variance in the fall Decoding 1 scores; for 13 
percent (.13) iri'^fie jfall I>ec^|Kjig 2 sc^es; fat'^A percent (.44) of the 
variance'* in' t lie Johrf T^st**fi^oi:es, and so on. ^ 

The ratios presented in-XaHle ^1 (degr^^ of freedom in parentheses) 



provide a 'statistical teefbf the niill^ hypothe§^ of ' np ptedictabiltty of \ 
each dependent variably (Achiev^enl: Test^*s^o»^ from the set o^independent 
variables (background variables). In (ither words,, is the mean s6ore< on' eaqh 
test the be^t a(Vailable estimator of how any given student will perform. 



srform, 
Dackgr^T 



or can we learn 'more about that p^rformaftce by. taking his or her background 

^ ... 

characteristics into account? The probabAity levels associated with these 
statistics are coded with asterisks. « ^ , 



'The background characteristic ^'high school .diploma" w^s not included in 
these fegression* analyses because it was found to be higfily colllnear 
with "number of years of education. "\ » * 
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'St^iyiardized Regiression Weights, Squared Multiple Correlationi and 
Statistical Tests for Background Characteristics- as Predictors of Pretest 
and Posttest Scirres; Day-School Sample ' \ 



Pretestg 
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Latfels Code: 



^D^; .-'Decoding^ Part 1 

0^5 : Decoding, Part 2 

L : Literacy Test 

: J : Joftn Test 

. M : Morano Test^ 

P : ' Oral Proficiency Test 



CT: ^^.Correctness Score 
JXa Comprehension Score 



-ST: Structure Score 
F 5 Fall 
St: Spring \ 



* ' * A prediction equation could be written "with the regression weights - ^ 
in*;any given column of Table 4.1 as the coefficients of th6 background 
variables r The actual standard scores of a particular student for each 
of these variably would be entered in this equation to predict .his or her 
standardized score on the corresponding achievement tgst. It can be seen that 
variables with larger weights ha-Ze more influenc^^on the predicted outcome. • 
The following ejmmpre will illustrate, this concept. We will use the numbers 
in the column l^eled PrCT, the c^orrectness score on the Oral Proficiency 
Test whic h was taken in the spring* The prediction equation for. this score 
after logarithmic transformatiw is: / ^ . 

' ^lo8^o(P-CX) = Z3 - .17 + .04 Z- + .233^ ■- .01 ?j . ■ 

- f + .03 + .07^Z^^.24 

^As will be^^seen shorfly^ €wo of, thet'' above weights are s^ignif icahfcly 
different from zero in tTie statistical senae. The three which are underlined 
in the P-€T column of Table 4*1 are large enojiigh to give some feeling for 
which background factors might affect correctness scores (P-CT). They are 
ag^, which has a negative\we|^Lght (- .17), status of job^in former country 1 
(.23) and amount of English taken in the U. S. (.24). In other words, student 



*who are jounger, had better jobs In their former country, SMhure taken moret 
course work to learn English aftei^ arriving in this country, will nave* higher ^ 
^predltted _CQrrectness scoxesi. . ' .. . 



"^'here are txgrp^other f^tjjires of the information in Table 4.1 whiclj 
shouL! be 'noted, jjtioso. regression coefficients which are significantly 
different from^zero in a statistical sense are marked with asterisks.' In 



'14S 



addition to"* significance in a statistical sense, however, it' is important 
to consider how much each predic*t6r variable contributes to the total 

^« 

2 ' ' 3 - 

^.predictability signified by R , without/iT^Ji^rd to sample size. The 

regression weights for 'tho,se variables[ which contribute at least ,01 to 

2 

the magnitude of the squared multiple .correlation, R , have therefore been 
underlined in Table 4.1 (i,e, , these- variables account for at^ least one " 
percenp of the total variance in the Achievement Test score). 

Implications of the Jlegressions of, Achievement 
c Test Scoreg'on Background Characteristics 

0 * 

y The R for prediction equation (1) is rather low (•20),^ This, e.quat^'ion 

is not a very accurate predictor of Oral Proficiency correctness scores. 
We may still speculate, hoxi^ever, that students with three backgroujid character- 
istics (Yotath, Good Former Job, Eng. in S» ) are likely to have higher 
correctness scores as long ^s we remember that the prediction equation will 
be highly in error in any given instance. By way of contrast, consider 

the comprehension scores (P-CP). "The prediction equatipn in this case is ^ 

^ " ^. * ^ ^ 2 " ' * ' ^ 

JLikely ^^^be a little more accurate <R = .33 for comprehension vs. .20 for 

correctness). In that equation amount* of English studied* in the U. S. has 

the largest weight* X^hree other characteristics are worth noting: age, — • 

whic^j again has a negativ.e, weight •.Za), status' of job in former country 

' (•28), and amoaht of English in the U. S.'(.34-). 

^ . - * ' ' ; 

, The similarity in the patterns of ^ regression weights for all three 

columris relating to the Oral Proficiency Test Js striking^From Table 2.5 

. /:/ ^ 

1 — ? " 1 

that^^some* we40its do not differ' Significantly from zero in a statistical 

sense does**not mean that we can^accept the null h^nppthesls and conclude 

that "^background characteri6tics«-dt: not relate to scores; all that is implied 

is thkf "eitTfxer our sample size is too small or our variables are too error 

prone,^^ skewed^ etc. to reject the fiypothesisj^f no relationship op the' 

basis of this one study. 



. .. - . /-^ 

we can see that these three alternative scoring systems for the same./set" 



items are very highly correlated' (before transformation) , as might be 
expected. The scoring for correctness' i^ most stringent, whereas ! that for " 
* comprehension is least stringent—it , is not unlikely that tjie lower pre- 
dictability of the more^tringent scores is due in part to a restriction J.n 
their ranges b'ecause f^w students got many items strictly c<y:rect. The 
logarithmic transformation helped tb^overdome skewing .0^ proficiency test 
score ^istril^utions due to thfe stringency *o£ scoring *an<i increased, their 
predictability* Consequently, the three 'alternative scoring methods yielded 



'•results which are even more highly e^quivalent after transformation thar^ is - 
indicated by the correlations giver* in TabLe 2.5. " : 

Background characteristics predict comprehension scores* befit (R = .33), 
structure scores less well (R =^*24) arid*" correotness^'.scores .most inaccurately. 

A reasonable hypothesis from these data, Kowever, is^ ^:hat .those students \^11 ^ 

* ^ • " ^ ' ' . ' ' . • • 

be more proficient in spoken English by the end of the year who are ^rounger, - 

had a higher status job in their former country, ^and have taken more EngliOT 

•courses^of pro'grams since coming toj^the United States. 



It should be remembered th^t predictive background characteristics do, % 
not necessarily "cause" higher or .lowfer scores. Age, for example, -does not 

necessarily make a person less -proficient. But age i6 associated with several . 

♦ * *^ • *"» ' 

other characteristics which, taketi together, may give lis sofie idea about why 

age is a ^negatife predictor of proficieftcy. Older students are more likely 

to nave a low status job in this, country, although*-they may have hac[**a " 

higher than average^ status job before arriving here. They ajjg more likely 

^ , . ^ V / • , ' . - 

tb* have had course work* in English ±n the previous country^-than in the*y. S., 
' • *^ * * . ^ * * , • // 

andi^are jno^e likely to have arrived from Cuba, and the Carribean than from* 

Western Europe. ^ - ^ * ^ ' * . ' \ V/ 



... ^ • 

Amdunt 'of .English studied in this country is more likely to be direct 

- • . • . I * ' • ■ 

influence on achieving proficiency. But, again, students who ha\{e had more 

work in English here are also younger, have arrived more recently, and have 

a ''history of higher level employment' both here and abroad.' They probably 

have both more need to speak English and more opportunity to do so.' Thus a 

ConiJ)inat:ion' of 'previously acquired proficiency and' opportunity to speak 

English is lil&ely to facilitate the progress of these students during their 

time at the Adult Learning Cejiter, , <7 

Differential Prediction of - Fall and. Spring . ' 

Achievement from Student Background Characterigtics 

Of tKe'five squared multiple correlatfo&s for predicting fall' tests from 

background variables, all but on'e are statistically significant ^t the .01 * 

k ' • • •? . / , . ' ' • / * 

# level. Note also that the multiple correlations are generally low^r ill magnitude 

and ^ of lower statistical significance for the spring tests. These ^results suggest 

^ ^ ^ I ? . * ' ^ ' 

that the role played by barffcground experiences in achievement is substantially 

. ' ^ r ' , - ' V ■ "■' .'^ 

dinjinished '.ov^r the course of the year, as tie w experience with English as a" 
< second language is acquired through classroom- instruction and extracurricular 
activfti*es. That students gain substantially in theiT av,erage .test scores hds 
; /already been seen in Table' 2.3. - *. • - ^ 

That background characteristics do help predict performance in the fall 
(R^ = .44 for the John .Test; = .47 for -the, Literacy Tfest) but that their 

.predictive utility is^diminished in the spring^^(R = .33 for Proficiency- ^ 

2 ^ ' -'^^ * • ' * 

Comprehension; R .25 for Literacy) is the most ^Important piece of infor- 

ttnation to note from Talkie. 4 <1. Only the coAprfehehsion.acore oil .the sgring^ 

. ' • ^. •* • •• •■ - . ■ 

-characteristics; This fall-spring distinction suggests that something ' 



beyond the students' background ^experiences prior to^ instruction, accounts ✓ 

for their final spring achievement. The 'itwrt question is whether or not 

«v ' • * 

we can identify any component of classroom interaction which accounts *for" 

' » - * 

the change in achievement. . . * % , 

There are a number of interesting relations apparent in Table ^4* beyond 

• V ^ • „ . . • , . 

those mentioned already. Consider thfe variable, amount of English sfudied ^ ' 
in the United ^States; one might expect that the amount of 'English any student 
has been exposed to or has learned will be related to his or, her measured 
achievement in the- fall. This predict>ion is borne out as can^be«see^ by 
reading, across in the line labeled "Eng in U. S." ^ote that the Literacy 'Test 
^ the John Test, and xhe Morano Test scores in the fall are 'all predicted, by 

** * ' # * 

■ -^his^variabj^e. This variable also .predicts the literacy ;Test scores and the 

^ - • . * / '* 

scores on the Oral Prof iciency Test iu the spring, as noted eavfi^v^ It 

accounts for not less than one percent of the total variance on all of the 

tests except -fbr Decoding 2 in ti^e spring. Obviously, the amount of English 

learned or studied while .in the United States is ,aii important predictor both ^ 

*of initial ^status and of inost aspects gf final, achievement. In fact, one 
' f ^ • / 

might .be suspicious of the validity or relitfcbilit;y of any achijevement- measure 

- * * • • ' - * 

which is not so preddcted^. 1 ' * - ' * ^ 

^ • ^ ^ ^ ,^ ^ : * 

Consider now the amount of .English studied, in the student* s former * 

, ^ ' ' t.jij , ^'^^ ^' . • . / 

country. One might suspect that English training in th^ country of origin* , 

is useful, but may or may not be as qualitatively good as ti;ainih§ acquired ^ 

-.in the United •States. But this variable also predicts some test scor^es^ 

as *can be seen by reading acros^ froffi "Forfiier En^g^, in Table 4.1, The 



regres»ion. coefficients fox the Dec<SPm^_2, 'Literacy, arid John Tests in ' 
the^^l are all signifiqontly' different from zero. Notice, however, 
that .the amount of English studied in the country ^of origin does not , * 

predict spring scores • f ^ . 

The difference in patterns of prediction for English studied in the 
U. vs. English' studied in the foriper counti^y is intriguing. It swggests that 



those 



individuals' who have studied their English in the former country 'ajfe ; 
more responsive to language experiences acquired during the interval between 
pre- lihd posttesting than are those Individuals who have studied their 
English in the U, S, That outcome is not surprising,, howev:er, when it is 
found that amount of »English studied in the previous country is our best 
avai]fS&le indicator of the occupational level acquired- in the United States* 

' V ' , , ' _ ^ 

(r = ,,29)* In this connection, note ffom Table 4.1 that -'Job in the U. S." 
^s one "background" characteristic which apparently has a higher relationship 
to certain jiosttest spore? than to any pretest score. Perhaps those individuals 
who^ have* attained high-level employment in Ihe U, S, have done so partially 
by. virtue of their prior English-language training in the previous country. 

This prior training also goes along with a higher overall level of educatiorf 

t- • , - * • ' - - * - > 

^nd, li^loyment itT^lie former country*, so %t is not surprising that these - • 



individuals are*^highly motivated to make use of their current English- 



language experience, both in the classroom and on the jbji. 

Note that^-the .regrea^iioivt weights for "age" are generally negative in 

sign. In this case^e see weights which are significantly diffei:ent from 



zejro in predicting both fall and spring, achievement scores, just as for . 



'^i^^ier empldym'Int: level and fingiish studied in the United Statesy^^ ^ . \ 



* * 'It is obvious from this analysis that a number of baclcground characteristics 
are significant predictors of fall achievement and a few of ttiem are of importance 
for predicting' spring scoree* Most ot^thfe multiple correlations are significant 
when all of the predictors are used in the regression . anal^si^, regardless .of 
whether fall or spring achievement is the criterion. 

Predicting ^osjttest Scores Using Pretest 
;i Scores in -Addition "to Background Characteristics 

TSie. next step in the analysis was to priedict the pqsttest scores using 
information about the pretest scores in addition to Information about back- 
ground characteristics. The* results of these analyses for * the. day-school 
sample are presented in Table 4^2. This table is- read in much the same way 
as Table 4»1. Across the top»of the columns are listed the, post test i^cofes, 
beginning with the two Decoding scores, then the Litefiacy score, and the last 
three columns ^contafii logarithmic trans formations of the three scores on the 
Oral' Proficiency Test* The predicted scores used in this analysis were all 
scores taken from tests administereoyin the spring, * . ' - . 

Down the lef t-harfd column of ^ the middle section^^of Table 4.2 are listed* 
thVlabels for'thfe pretest ^coreg; included are the tw6 parts of tl^6 Decodirfg 
Test^ the Ll^^cy Test, the John Test, and the Morano Tes^/. The 'first line 

?*■ ' . ■ ' ' - ^- -^x ' , ' . , ' - 

of this table gives -tHe squared multiple correlations resulting from prediction 
of post test 'scores from background characteristics; these numbers are repeated 

-frbia th^i^rlght-hand portion of Table 4.1. . The entries in the line labeled, 

.\ ^: - v/- ..... . / , 

*t'*R with Addition of Pretests," Van be compared to. the corresponding entries 

Un fi^st line. Note under Pj^^S that R when only background character- 

istlcs are used is .25, but R Increas^p :tQ (.60 wheA the pretest information 



TABLJJ 4.2 



^ Standardized^ Regression Weights, Squar/^ .Multiple Correlations ^nd . 
Statistical Tests'for Posttest Scores /Predicted fronf Pretest Scores 



Statistical Tests' for Posttest Scores 
Adjusted for Background Character 



/Predicted fronf Pretest Scores 
sties; Day-School Sample 
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-U-38 - ' > . ■ - 

* ' ^ ■ ' ^ 

is added. In gtjier words, if" only background charjtc^^eristics ar^^sed to ^ 

r 

predict posttest scores, then 25 percent of the variance in the' Decoding 1 
posttest is account^ed for. But an additional 35 percent ^-S accounted for 

if information from the pretest scores is added. Note, for example, that 

2 ' • '"^ 

the R for predicting transformed Oral Proficiency comprehension scp^res 

from background characteristics is* only .33, but with the addition of pretest 

information it becoijes .82. ,A similar pattern is apparent for the other two 

scores of the' Oral *Proficidncy Test. Thus, the students' initial jproficiency 

plus, some infornji^fcon about; their, background characteristics accounts for 

a substantial proportion of the variance in the comprenension scoreSo In^ 

summary, to understand how much the prediction is improved by adding pretest 

. : ■ • .2 
data to backgiTDund information, simply^ compare/:he^ R j.n^ ^he first line ^wlJth^^^ 

the xn;]^->third' portion of the table.. "V * 



^ .^It^hould Jbe noted that most of the squared multiple correlations for 
backgrwmd characteristics pljis pretest scores are substantial <.60 to" .82). 

"llllreover, the increase Jii all of these R^^s except for part two of tlie Decoding 
Test are highly significant when compared to prediction from background^ ^ 



characteristics alone. 



i» - , Except for part two of the Decoding Test, all of the squate3 multiple ' r 
' cotxf^atS^aitS posjttest scores with background and pretest variables are 

^ - t < ' * ,t ^ > ^ ^^i:-^ _^ ^ • - - ^. ' i 

. ^ ^ at orv kfcove .60.'^^^r^^)ni;TaU^ we can s^e that the posttest reliabilities 

for ali; e^Kcept'* tfecBd'iag^ii^^ .88 to. .94 (Decoding 2 'has j^wer ij^t^mal 

. . •consistency, y^eidin^Ja^^el^^llty of .77). If we take these* reliabimi^^^^ 
to -mean thatx^ roughly 90 percent of the variance in^most posttest scores "is 



reliable., tften it can be said th^t W^r. . 60/90 ,ot- two-thirds of .that variance 

' • : •■" . .*\ 

-ERIC ( 




can be accounted for* by the background and protest variables which we have 

^ measured. This jleaves up to one-third of the reliable ppsttest^> variance 

which might be accounted for by classroom experiences. Howevar, we must 

keep in mirtd that we do not have separate measures of classroom experience 

for each of the students in any classroom — only contrasts between clad^rooms* 

This, mean's that we cannot expect' to increase the accuracy of our prediction 

to the limit of the reliability^of posttest measures. on the basis of class- 

room experiences. Even if classroom experiences are highly influential, their 

effect cannot be detected unless there is sufficieSYit homogenity among students 

within classrooms with respect to these experiences* We initially planned to 

use attendance, as a means of assessing individuals differences in classroom 

"experiences, but the data on attendance did not prove helpful because of " 

.llttle 'dacumelntea variation in 'attendance* ' * ^ . - ^ ». 

• * * • ^ • . - 

The consequence of all this is that our concern in assessing the impact, 

' * - ' - * 

^f cla^sjroom experienc'es upon final achievement scores must focus' primarily 

upon, the results of significance tests Comparing between vs* within classroom 

.variation) rather than upon any absolute increases obtained in R ♦ 

Before idoving to the assessment of .the impact' of classroom interaction, 

contrasts upon final achievement beyond that predicted from background an(|« 

pretest variables^ it is in' order to notice several' features of Table 4*2. . 

As far as achievement is concerned, the only, predictor of Decoding 1 in the 

spring is^ the corresponding pret^^t score. Spring Literacy scores, however, 

re seen to' relate both^to the Literacy ^est and to the John Test it! the 



: StudBnts who fall feo^attend regularly are noGacontinued in the program. 



^ fall. The spring Proficiency measures, which are highly ijitercorrelated as 
^ we mentioned earliei*, are largely a function of John Test performance in the 

. • ■• • ' ' 0^ ' 

faA.1 but also appear , to reflect Decoding 1 performance in the fall. • 

- • . . ' ■ * : •■ 

.Posttest Scores Predicted from Classroom tnte^act4,on, with 
Adjustment for Pretest Scores and Bapkground Charaateristics ^ . * 

Table 4.3 presents information from the regression analyses <rfor the 
f day-school sample in which .posttest scores were predicted from classroorar 
interaction contrasts, pretest ^scores, and background characteristics* This . 

table is organized in the same way as Table 4.2. 

• * , 2 

The fir^t line of Table 4.3 gives R from the regression of posttest 

scores on background characteristics and pretext scores, as seen already in 

i Table 4.2., , Next are four lines with Romarv numerals I through IVj within 

each lin^ can be f^h«;^ the standardized regression weight of the toriesponding 

transformed canonical variate (see Chapter Three) in the prediction equation 

' for the achievement posttest indicated at th| head of the column. 

• - ' ^ 2 - 

* * . Tlie^sixth.-line in Table 4.3 gives R once the four transformed canonical , 

- - variate, classroom interaction contt*)asts have been included in each^ prediction 

equation. Again, lines conjtaiijing the initial and final'R values can 

be compared. , From this comparison we can see that the addition of classroom 

* . * ' 

. * interaction contrasts adds little in .an absolute* sense to the accuracy of 
t -'^'preiLtition of posttest scores (ihcrea$es in R .vary from .04 to •09). 

/ . I Despite the relatively small absolute increases in multiple prediction. 

^-'i^ ' . ..'ii 



equation accuracy obtained by supplementing backgrpund and-.ttretest infon^tion 



^^i;., ^ with, classroom interaction' information, it can be seen from^ the F-test results 



in Table 4,. 3, that the .increases in predictability obtained^qannoj: be attributed 
'to chance alone. / | . c iv ^ ' - 



■'■•'o' ^ . V \ • 158- 




- 141 - . ; , 

: table 4.3 . 

■ .. ■> .; ■ . ■■. • 

. , ^ Standardized Regression Weights/ Squared Multiple ^Correlations, . * 

and Statistical Significance Tests for PosttBSt^Scores Predicted "T" • . ' 

from Classroom Interaction Coivtrasts,- Adjusted fof^5tudent Background 

I Characteristics and Pretest Scores; Day-School Sample . " 5 
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with Addition 



of Classroom Inte?- - . . • . 

acti'bh Contrasts ' .65 .^28 ' .66 , ^<7lr> ;<.85 .51 

/ , ' \ ' ' ^ ' • / ^ » 

F (4, 62) Test of • 

Information . + .\ + + * . 

, Increase^ ^''2.26 2.03 2.35 " 2.74 ?.73 3..91 



i 



y^^- " *p- ?c . 05 

p< .01 
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we pointed out earlier^ the clas'sroom interaction contrasts apply 
equally to all individuals within each , classroom /"so great /homogenfeity'of 



*~ experiences would be required in order for , these ctmtrasts to yield a large 

absolute increase in the accuracy, of prediction or posttest achievement.* 

t The F- test results relate, hpwever, to relative increases in accuracy of 

• ♦ 

prediction; i.e.^ given that we have already reduced uncertainty about gost- 
test performance quite considerably from knowledge of bacl^ground. character- 
istics and pretest performance. TJiese relative increases in the accuracy of 
prediction are substantial ^enough, even ^iven our squall sample size, to support 
further exploration of hypotheses to the effect that the c^lassroora interactioa 
^experiences measured in this study . contribute tb final •achievement abo-v^e and 
T)eyond ari the student background characteristics and initial abilities which 

we measured; • , , * ' • 

* Interpretation of the standardized igegression coefficients in the columns 
of Table 4.3 requires reference to the earlier interpretations g^i^en ,to «the 
four trans tormed^^canonical^variates (see Chapter Three). Recall that 'the 
firdt two classroom interaction contra'sts reflect broad stylistic differences 
among teachers/ The negative pole' of each contrjast relates to rather ftighly 



• structui^'ed class or grouproriented activities. The prevalence of "teacher 
modelrstudent practice" Interaction^pSttems on the negative pojleS suggests 
iraplementa^iO^of the audiotlingual method of instruction and seems to be 
, related to teacher backgrou^^d characteristics (see end of Chapter Three).. ^ 



Possible relationships of 'the classroom intetactp.on contrasts to teacher ^ 
background characteristics were inferred ftom. a comparison of Eigurfes 3.2* 
and 3.8, coupled with a canonical correlation analysis which is^ not reported 
here. . • 



< ■ - ■ - •••• ■ . ■ ' ■ ' •-. • I 



The presence of the "Free" and "Other" patterns of' clas^rdoiH i^eraction on 
the positive pole of tVie second transformed axife leads us to think of this 

*variate as a contrast between the silent way and 9.i|dio-lingual methods, ^ « 
The flaxible-i individualized, supportive, and encouraging aspectis of the -> ♦ 
positive pole of the first Nixis suggests implementation of a rather eclectic 
version of classroom interaction. This eclecticism s^ems to be characteristic 
of thos^ teachers whp have worked mainly wit;h adulta and have little formal*^"" 

^^tra^ning*; perhaps the^^ treat the s'tude^tts more as peers than do other teachers 
and thus rely less upon preconceived notions about "systems" of instruction* 
The third axis relates most , directly to' the occurrence of "student-student 
feedback" during instruction* The fourth axis can be ignored for the moment, 

Jne most; striking aspect of ,the patterij of standardized regression 
coef ficietits presented**in\Table 4.^ is that it appears that those features 



of classroom 'interaction which ^are associated with higher than would o^her-. 

I /'•■'/' ^ " 

/ wise be expected posttest scores on the 0?:al Proficiency Test (i.e., axes 

J/ ' ' I I ' ' - i ' J i ^ ^ . 

I and EL) aire associated wi^^h lower than would otherMse be expected posttest 

' ' ■ ^ . ' / ^ / / . / / 

scores on the Literacy and Decoding Tests (compare the weightiF in the , 

left-hand columns of Table 4.3 opposite I and II .with thpse in the right- 
ifand columns opposite I and II). *Ef we were to take these coefficients 
as the basis for .formulating hypotheses alifeut the ef fect^iveness of individual- 
ized* instruction vs.. group instruction, tKe silent way vs. the audib-lingy^ 

, methoa, an open and ""supportive vs. a highly organized and directive classrdom 
climate, etc. (however one sees, fit to interpret what^vis in common to the 

* finest two '^fcransforme<l canonical, variates), then it would* be in order to 

expect the former (individualized, supportive, silent way) to lead to increased 



to lead to Increased literacy/ decoding skills. Moreover, there is the 

' ' X' ' ' * ' ' 

implication that what might lead to' higher than would otherwise be- expected 
finajL achievement in the oral proficiency domain might/lead to lower than 

^ ^ y . J / ^ ^ ' • 

wotild otherwise be expected final achievemeirt in the -literacy/decoding domainf 

It appears, ^therefore, ' that any hypotheses based upon the achievements- 
test regression weights on axes I and II in Table 4.3 mufet involve a tr^de^ff * 
in terms o^'-tihe achievement; goals to be stressed — what appears to be iticremental 
for orkl proficiency appears to be detrimental for literacy/deco(ilng, and vice 
versa. The same. can be said for axis III, "student-student feedback," since 
its occurrence in a classroom is'predictive of lower than migh't otherwise b^e 
expected oral proficiency (comp^rehension ^mainly ,• as seems quite plausible upon 
due reflection), but higher than might otherwise.be expected literacy/decoding.^ 

' As ,fpr that ubiquitous, bipolar contrast in classroom interaction pa^tdrns 
(whether at the level of irfdivldual episodes, sessions, classrooms* or teachers), 
"teacher direct-student i;ead and/or ask que^st^n" vs.* "teacher ^question-student 
answer," we again see it playing an important role. In this caae t)iere is 
no apparent trade-pff , however. On. the contrary, it is suggested quite 

/ I ^ J : • ' / • ' /. " . ' ^' I ' • : 

.clearly that those classrooms in wlvich students are directs to read are also 
those classrooms in whiph we can predictlthat the students will .achieve 

■ > ■ ' ^ - -2/ ' . ■ , ■ ' 

hi^lier ,fhan would otherwise be expetffed, on the Literacy and Decoding posttests. 
The contrast seems tp ,bear no r^al Relationship to oral proficiency, suggest;ing 
that nothing is to b^lost and literacy/decoding skills may veil be gained 
through classroom implementation of the "teacjier direct-student read apd/or 
ask questioixs" paradigm (line labeled IV in Table 4.3). °^ C t 




PE ANALYSIS OF DIFFERENCES IN STUDENT ACHIEVEMENT BY CLASSROOMS ^ 



Notice that we have been rather liberal about discussing t^ie possible 

'implications of^.the regression weights in Table 4.3, as far as stating 

j» » 

hypotheses aboiit the effects of teaching styles is concerned. Furthermore, ; 
we have relle^^^'^s^nly upon the signs of the weights ^withbut regard to thei,r 
magnitudes or their departures from" zero in a statigtical sense. 'This approach 
±s perfectly acceptable, as long as we indulge only in' the generation of 

hypotheses,' and not in the acceptance or, rejection of such. ^ ' 

* - <« • ' 

It is 'not possible to confirm causal inferences in ap exploratory survey 

of naturally existing populations (e.g., classrooms]^, eyen where statistical • 

significance -of relationships is^^gid^lJy adhered to as a basis for entertaining 

hypotheses. We ca'n only rely upon statistical procedures to help clarify 

^p{)atent relationships and to arrive at plausible models of a quasircausal 
f * ^ • ' ^ • * - 

nature whicfi might later be sut>jected to experimental* refutation^ ' ' . 

One basis for evaluating a mathematical model is the degree to which ' 

it "saves the appearances"; i.e.,, the extent to which it yields a closer fit v 

to thef' manifest data than other models. which are comparable in complexity 

■ ''7 .'■ •(/' • / ./ .•'■•/ • •// •■/ ;'/■/ • -7 

(e.g., in the number of parameters^to be .estimated from the d^ta,^ ^suclj as ; 
regression coefficients)^ A* plosely related Issue is the .question of how o * 
much improvement in the-fi-t of the model to obse^|rved. «at4 tan be achieved 



through relaxation of restrictions on parameters in the liiodel which have 
beien fixed a priori, and thus ajre not estimated. from ''the data. This is the 
actual basis, of the 'stfrategy' which, w^ have employed so far in comparing " 
regression analyses; i.e.,. we proceeded from the simplest model, in which 

v^..,-f ^ ■ ■ ■■ ^ ^ i • - . ■ 

only, the grand meaa was used as an estimator Qf posttest performance, 

,• y ' ' ' -L \ •■ , • !• 



through successively more complicated models • Iiu going beyond estimating 
medn posttest performance alone, we first allowed_J:he regression coefficients 
'for backgrpiund characteristics to d^art'ffom their a priori values of zei;o,- 
then the coefficients' fbr pretests, and, finally, ^the^ coefficients for the 



ipor 



foift: classroom interaction contrasts. - — . ' 
* ^ In^eadb^Ncase of successive relaxation in £ priori restriction| upJn ' 
regreission i2oe^ficients, the F-test of increase in information indi<:ateJ * 
whether or not .enough gain in accuracy of prediction (fit Of the^mafbematical 
model to observed dafta) was achieved to justify the associated S::.oni^licat ion " 
in the^mod.el. / ' * <^ ' r > * ^ ' 

^- • • • -C 

'Predicting Posttest Scores Using All Possible KC6n,tfasts Amqng Classrooms 
' With AdjustatBnt for Stajdent Background Characteristics and Prerte'st Scores . 



We can now carry thfe model relaxation process one step furt;her than was 
finally done with the data* as presented -^n Tdble 4.3, by allowing parameter 

estt^mates for any possible differences between CTassrooma or .teachers. Thus ^ 

' ^ \ , • 

we* nd longer restrict ourselves to the four .classroorf contrasts derived from 
observations of teacher and student interactions over the cours^of the stmdy. 

If the/expanded (relaxed) model result $ ^in^ niucH Impffiv^ment In fit, to, the> j , 

^ - ' ' • ^ f f ^ ' ! ] ^ ' ft \ - r*'' ;^ (H^ ' 

observed dat^^ then we muat conclude that what was most reliably ^epordedin 



the classroom observation Jhase of the study, taken together .with pretest 
and backgrounds-Characteristics, is not adequate to account for 'Variation in 



posttest performance. What, emerged from the*factor analyses o£ classroom 
interaction data inv^Chapter Three seems to relate more to variation in the^' 



structure. of teacher an4 student interax:tions than to variation in the 

' ^ - ■ ...... \ - • • 



• .■ . .-, ^ r-\ ■ ■ . ■■ ; . _t 

content of those interactions* Variation dn this, content is one" possible ' , ^ 
/'Source of classroom dif fe:jrences in achievement which we may not have assessed ^ 
adequately. * • . '-..^ 

•In the first line of Table 4.4 we have presented a^gdin the'squared 

r 

multiple Correlations qf posttest variables with background and pretest* ^. 

scbre^. These entries are. the same as those seen on the first line* of -^^^ 

2 

Table 4.3. Tlje second line ^of Table 4,,4 contains comparable R .values. ' 

TKese R 's were obtained by entering a complete -set bf ^3 contrasts (one for 

each degree of freedo^m) among classrooms into the predic^on equation.^ The 

2 • * 

F-tests Qf information-'increase associated with these R values are all 

statistically significant, indicating that dkere aye 'real differences in 

* • • • * ' 

student posttest achievement among the vario^us clas"*srp6ms in the study, even 

after adjusting' classroom means fpr preexisting differences in %tudent back- 

ground characteristics and pretest achievement (assuming homogeneity of 

y * » 

wi thin-classroom regressiofi planes). 

We have already seen indirect evidence that the original four classrooraT^ 

interaction contrasts account for some of the differences in adjusted class-' 

room means on the posttest achievement measures (table 4.2)^**2g*J4o-V^^*^re in a 
>^mosition, how.ever, to partition adjusted posttest variation* aiiong classrooms ' 
. into one component which can be attributed to the four classroom interaction 

contrasts and another^ component wh|ch mx^st be attributed to diffCTen'ces ^i?iong^ 

classrooms which we failed to measure reliably in the observation phase of the 

• *" ^ ' . ' ' 

study. " • V . , ^ 

The last four lines in Table 4.4* are addressed to this Issue.' They ' * 
contain information about the separate .contributions of our original four 



0 

L 



TABLE '4.4 



Regression Analysis Results, when Possible Classroom Effects are Entered , 
Compared to 'Results when"* only Four (fla^sroom Interaction Contrasts are Entered 
' . . ^ Day '^School Sample . • " 



R from 
Background ' 
and Pretests. 

2 

R from 
Background , ' 
^ Pretests and 
full set. of 
Classrobtim 
-m, Contrastsi*^ ' 

Overall 
(13, 53) test' 
of Adjusted 
Classroom 
'^Means. 



' III 4/ 

, . - Ji . 

— ' IV\ 

%-Reductlbn in 
Residual bum of • 
Squ^ires per df. 
Canonical* Classrobta 
Contrasts • • 



F (4, 53 )v Contribution 
; of CanpMcal Contrasts 2,53 
« « 

7o Reduction iii Residual 
Sum of Squares per ^f> 
•N ^' Isemain^ng Classroom- 
X -Cont.rasts . , ' , 16, 7 

: • ^' ' . ' ^ I 

; FX9>^330 ContriU#ion . 
of. Remaining, Contzrasts ^ !• 70 

/ ' . + p< 

.4 * p< -05 * • . 

** p< tOi 

. ***,p< .001 ^ 
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8.0 



2.26 
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classroom interaction cbntrastf. (transformed canonical variates), as weLl 
as . the nine^possible^ remaining classroom contrasts, t6 variation in adjusted 
cla^sr.oom means on the -respective pos£tests» , 

Information, in this section of the table has been presented in two 
different ways in order to help the reader get a feeling for the relative 
si^es of the effects involved as well as their statistical significance. 
The 'X^an percentage^ reductio^ in residual sum of squares per degree Sf 
freedom" value for each collection of contrasts gives some feeling for the 
size of effects involved; for example, 4 x 10% + 9 x 6.7% = 100% of the 
reduction in sum of squares for the ^^^-S posttest. . The associated F-tests 
indicate whether the additional parameters estimated can be justified on the 
basis of improved fit of-^the model to the data. 

, - ' From the last four lines of Table 4.4 we can learn a good deal abou^ 
thfe possible bases of differenced among classrooms on posttest performance, 
beyond what is accounted for by the background characteristics ^nd pretest 
performance^f the students involyed. Only for the Decoding 1 posttest is 
it safe to conclude that the fl)ur classroom interaction contrasts derived 
from. canonical discriminant function analysis of the classroom" interaction data 
are sufficient to account for all classroom diff fences in adjusted posttest 
performance. The regres'slon weights in the. mid-section of Table 4.4 reveal 
that, the relationshi]J of Decoding 1 gerf^Drmance to classroom interaction 
"stems primarily from contrast IV, "teacher direct-student read and/or ask » 

■ - ■ - ^ / 



question" vs« "teacher question-student answer," as was sugge-sted by the 

* 6 . 
results in Table '4.3. . ' : • 

Notice in studying^ the regression coefficients of Decoding 1 on classroom 

interaction contrasts in Table 4,4 the fact that the second contrast ^ (II) h^s 

a positive weight'. This'result contradicts the tentative results given in 

^able 4.3, but' the coefficients in question do not differ significantly from 

zero in either case. In'only one other case does inclusion in tlje model of 

addit4.ohal contrasts among classrooms change the direction of impelled effects 

• ' • ' " 

• r ^ • 

from those given in Table 4.3; but again, the coefficients involved are not^- 
significant (regression of the Proficiency Structure score on Contrast III, 
"student-student feedback"). ' . - 

From the last two lines of Table 4.4^ we can see. that the Literacy,- 
Proficiency Structure, Decoding 2, and, perhaps. Proficiency Correctness 



* The regf^^ion coefficients giveti Xp. Table 4.4 are in raw score fora and. 
^PP}5-y to transformed canonical classroom interaction contrasts which have* 
been normalized in the .metrtc of the classroom cell counts. Seven of the 
remaining nine contrasts referred to in the table were' derived by taking, 
in successive order, those seven original classroom interaction factors 
which had maximum residual betweeAclaSsrooias variation after partialling 
out any effects which could be attlScbuted tcf -the four "Qriginal transformed 
canonical variates. The reason for this step was to obtain, in successioii^ 
and to as large an* extent as possible, stable classroom differences. The 
final two classroom contrasts were actually teacher contrasts since they « 
were^the .projections of. the day-school teachers (identical projection for ^ 
different classrooms taught by the same qeacher) onto. the axes o,f the ^ 
multidimensional scaling solution presented in Figure 3.2'. The nine new 
contrasts were transformed to be orthogotial to ^ the original four classroom 
interaction contrasts, as well as to be mutually orthogonal in ^he metric 

of the 'classrpom cell counts (in succession, using Gram-Sch£i^^ orti\o^Onali- 
zation).' Since the* choice of these additional contrasts lis arbitrary from 
the point of view of overall fit, several other alternative baaes were 
explored in hopes of gaining insight into the natur^ ot the classroom 
variation involved. Since nothing much came of t^ese;, explorations in the 

* way of explaining .the relationship of classroom interaction to student per- 
formance, the results will not be presented herie. S.uf f ice, to say that 
there was some evidence for reliable differences in classroom interaction 
factors beyond that measured^by the tirst fout canonical'*variates, but more 
evidence tha^ teacher background characteristics might contrijjute to student 
achievement (experience in adult day-school ESL for pr6ficiency; •education 
for literacy)," , ' , * \ • 



' ^ ^ . \ • • • - ^ 

pdl^ttests all show clas'sroojn-ip-classrootn variation in adjusted means which 
must be attributed^ t!o something beyond the reliably measured aspects of 
classroom interaction embodied, in the four trans foiTmed* canonical variates. 
This fact provides justification for looking at mean performa'nce data for 
the individual- classrooms in order to Seek an' explanation for their differences. 



' ^ Simple Comparison of Pretest ^nd 

Posttest' Performance in'Day-School Classrooms, ' ^ , 

In order to study the mean performance of individual classroonf^ we reverted 
to simple analysis of covariance procedures in which each pretest was ,the so4.e 
cbvariate for its respective posttest. An exception: in the case of the 
Oral. Prof iciency Test/ only the correctness sc6re was investigated ahd both ^ , 
the Defcodi^g 1 an4 the John Test were considered, individually,' as coyariates. 
iPor the sake of completeness, the Decoding 1 Tests were also studied with-'simple 
^^iysis of covariance', even though there is no evidence in Table 4^.4 for* * 

^ • ^ ' : ' * / ■ - , ' . - * . 

classrooin differences beyond the four classroom interaction contrasts. 

V - . • ^ ; / . ^ , . ^ - 

Figures 4.1 through 4.5, are plots of tkfe 14 day-school classrobra means 

on pretestj^cpres (absci^ssa) vs. posttest scores /ordinate) for the Decoding J., 

.J)ecoding, 2, .Litieracy, and Proficiency Tests (the -proficiency means are in log^^^ 



units and are plotted against 'John Test scores in Figure 4.4 and fall Decoding 1 
scares in Figute 4.5 because the Proficiency '^est l^ad.npt been administered as. 



a pretest). Individual classrooms are coded 4.n the figures by ^the same letter^ 
and number system used in earlier figures. J * .* , , 

« The line drawn through the points in each figure is the pooled-within- 
classrooms regression line, and m^y be taken as the point oi Reference for ^ * 
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Decoding 1 Classrot)in^ Means for Fall and Spring Administrations, 
Day-School Sample (Pooled .Within-Classrooms r ^"'•&9; To/tal Sample 'f ~ .73) 
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FIGURE. 4^3 



^ Literacy - Classroom/Means for Fall and Spiring' Administrations, 

Day-School Saimple (Pooled Wl^thin-Classrooms r - . 59 ;^^ijtai«" Sample -r = .71) 
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FIGURE 4.4 



^ ' -l^g^Q Sprlh'g Proficiency Correctness Cl^assfoiom M&an 
Scores x Fall John' Test Classroom Hean Scores, Day-School 
Sample (Popled Withln-Gfoups r = /28, Total ;Sample\r^== .77) 
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FIGURE 4.5 



Log^Q Spring Proficiency Correctness Classroom Mean - " 
Scores x Fall Decoding 1 'ciAssroom M^an Scores, Day-School' / ^ . 
Sample (Pooled Within-Groups r = .44, Total Sample r -^^.67) ^ 
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• - • ' ~ s • • 

determining the adjusted gains- or losles of individual classrooms if jthe 

• • < ^. 1 .- - ' ' - ■ 

withln-ciajss rooms regression, coefficients are assured* to'be homogeneous 

' ' ^ ' • . • ' ' ' . ^ " 

enougfr to justify^ pooling. Whether or not pooling is justified is hard to 

ascertain with the limited data available because class dizes range oniy ' • 

from 1 to 10, as indicated in the inset in Figure 4.1. . Since some/of . the 

wi thin-classrooms regression coefficients were negative for all. analyses 

except Decoding 1, there is'^reason to suspect that> the assumj^tion of homogeneity 

of within-groups regressions is untenable. \ . / 

. Diff-ereuces in Achievement of Literacy 

• Referring bacK l^o Tal)le* 4-4, we se^hat ,the Literacy Test shows evidence 
of 'much, variation amoVig adiusted, cla^sroom^psttest means which cannot be 

' \ \' ■ ' ' ^' >^ [ V 

accounted ^^f or by r^fei^ence solely to the four Veiialjle^ classroom intens^ption 

•ontrast^.l From Figure 4.3 we see %hat this yariatiori is probably due to the 

. exceptionally good posttest perfoyroance of ^tj^ level I'^sSudents in classroom 

. These students begin the year at the very /lowest J^evel of performance 

^ ;* .4 

on the Literapy Test, Uit, by the end^ the year, the^^are as literate as^ 

N stMents in several levell 2'^rid level 3^ classrooms. This performance contrasts 

. • ■ ■■ • ; ■ 1 " ^ 

mirkedly ^Jith. that of students in classroom Bl, where spring Literacy Test 
, perforfhanc^ is essentially unchanged from »that in the fall. Obviously, teacher 
. is increasing tlie English language- literacy of level 1 students more than 

■ • other teachers at that level. A similar copclusioA holds for pecod\ng 1 in 
, Figure 4^1. * - * . 



Differences in^fc|.evement of Decoding Skills 



The Deccrding 2^ test has been revealed to be a bit. Anomalous in the foregoing 
analyses in that performance on the test is 'no^t highly predictable from a 
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knowledge of student background characteristics, pretest performance, or 

classroom interaction experiences* From Table 4.4 we see, moreover, that * 

there are real differences among classrooms in terms of adjusted Decoding 2 

posttest means which cannot be accounted for by reliablje differences in 

> > ^ ^ * \ / * 

observed classroom interaction patterns. Figure 4.2 helps clarify what ' 
might be going on With Decoding 2 scores, which in part measure how well 
students* can pronounce or unde:?stand ^he ^pronunciation of English words'. ^ -It 
seems that cert;ain classrooms in. which studer)ts have the poorest relative 
grasp o^^honeme-grapheme correspondences in Jthe fall are the very classrooms 

in which the students haye achieyed a relatively superior grasp of these; 

• . ."^ ^ 

># 

correspondences by the end of the schcrol year; and vice versa . One obvious 
way in whi^^h we can interpret these results is to hypothesize that teachers 
in some' sense overreact to their students' initial aljilities in English 
pronunciation: if these abilities are initially minimal, then improvement 
is sought; if .these abilities are initially superior, then otiher aspects of 
performarice are emphasized. ' \ 

' ^ ? 1 i i V * r • ^ ' • ^ 

DfcLfffirences lln Achievement* of Oral Proficiency ^ » * ] ^ ^ 

'As for the Oral Proficiency Test* structure scores, little is revealed b7 



'inspection of Figures 4.4 and 4»5 except t^at the achievement *of leyel 1, . 
students in classes Fla and Fib is lo^er than might be expected. Nbti^e, 
however, that neither the, John Test nor the Decoding 1 Test is an optimum 
"pretest" for - oral .ptoficienc^; ^fdrtuAately, the Oral Proficiency Test 
itself was 'nd^j^^^elopred^ui^l ,a^ter pretesting was completed.*-*^ 



^ ^ The tendency for 'level 1 classes' to lie below the pooled-within-groups . 
regression iinres in most of the figures (except 4.2),* while level 3 classes 
tend to lie above the same line, indicates the need to include more than one* 

covariate in each regression analysis in order to account adequately for 

* ^ - " ^ \ 

preexisting differences among classrooms. This was our reasoning in tt^e 

decisiQn to use all available background and pretest scores' as covarJTates 

in the regression analyses* reported earlier. Although the. relationship's 

Qf these predictors to posttest perfbrmancIT were thoroughly explored and 



xiiscuss^d in ponnection with Tables 4.1 and 4.2, we had not yet faken into 
account the grouped^* structure of the data (students within classrooms) as 



ructure of the data (^^dents within class 

\faj^. done.^with the anaj.'jrses presented in Table 4.4. ^ j * , 

Once all 13 possible classroom Interaction contrasts are 'included in 

the analysis, moreover, 'there can be changes in the regression weights which 

^ . ^ . ' 

must then be attached to background and ^trete^t variables for optimal pre- ^ 

, diction of postt^est-perf ormance. 'These final ifagression weights -characterize 

a mo de]^ in' which the grouped structure*,of the^data is taken properly into 

account. They are. of interest because they should* generalize to pr^lct , 

-variatH>n among stud qpts within *aj broader, r*^nge o¥ •blassrooiifc than th|p limited 
' - ' . ^ If = ' / ^ ^ * , ' \ \ 

selection .included in thip study. In/other words, once all the effects class 

room differenced are eliminated, we can Req.eralize about variation among^ 

'I ■ ^ " 7 

students within any classroom Which might be »imilai?vto those studied here J[ 
• ^ ^ ' >^ ' ■ 

(provided ^e Accept the assumption of homogeneity yf-:fegression planes). 

ji>oled Wi thin-Classrooms R^gressiotis of Posttest Achievement 
. Scores on Stydent^ackground' Charat^teristics and Pretest Scores 



Table '4.5 presents standiardized regressJ.on coef f j^elSnts, of spiflng pt 
scores on student background and pretest variabLas with«the effects of classroo^n 




TABLE 4.5 •' 



• * ' ^ - _ bj,-s ■ D^-S 

''^Sex (F) ' . -.12 r-,j|;3 



Age • -.11 ^10 

Time in U. S. -.05 * .04, 

Foxoer Xob - -IL -09 

Job in U. S. ;12 * .10 

«* ■ ■ — 

• • + 

C-Origia , -.09 ' .18 

Years Education . 01 .14 



Former Eng. . ^ " ' -.06 

Eng. in U,. S. .05 • .04 



WitKin-cl ass room ^ 
i; Day-Schooi Sample 
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differences eliminated (day-school sample). This table Is a companion to 
Table 4.4 since the results were obtained in the same overall analysis. 

• . . • - ■ - 

We have* held off discussion .of these cde£fiqients until, now 1)ecause the 
coefficients, presented are analogous to the pooled-within-classrooms regressions 

(correlation, actually, since they are standardized) discussed<*in con^iection**" 

* . * * - <^ 
with Figures 4.1 through 4.5. ^ 

. * " - '-^ ' ^ ' ' ' 

When the standardized p'ooled-within-classrdoms regression , coefficients 

in Table* 4o are compared to corres|)onding entries, for the total sample in, 

♦Table 4.1 (^osttest section) or in Table 4.2, several noteworthy. 4iscrepancie6 ^ 

can be observed. It is not surprising that the large role played by initial ^ 

John test scores in the prediction of total-sample vi^iatidn in Literacy and 

■. . • ' , ■ • V- * 

Proficiency posttest scores is substantially digsinished once classrobm ,dif ferenceg 

* \^ . - " • - ' ' 

are partialled out. A" major distinction Uetween classrooms is the piroficlency 

level of 't'he students, *as. measur'ed by John Tesf-s^ores, so wi thin-classrooms 

, ' ' • ' ./ . , . ''^ ^» . . ■ \ y ■ 

variation *ia John Test scores is mino^ relative to between-clas'srooms varia'tion. 
Notice that. the residual, within-classroom' variation on John Test scores only 
retai^is predictive utility for, spring Ora^ Proficiency Test scores (Table ^.'5). 

% .have ^already noted Ae fact;, that between-classroom regressions tend ^ j 

: ' - . * • / • 

to contradict within-classroqm regressions fpr t^e Decoding 2 Test, as can be 

*seen in Figure 4.2. A compa^-ison of^ Decoding 2 standardized teg2?ession^'^ ^ 

coefficients oh pretest . scorers' in Tables 4.2 and 4.5 will .reveal the outcome 

of this contradiction-: '*a lt>w weight on the Decoding. 2 pretest, in the tot^l * 

• • * ' - ' ^ -<i * 
sample even thotigh that score turns oufc .to have substantial predictive " s •* 

< ' ^ ; . 

.ability within classroomsv . . /.^^ 



Notice, from the,lowet section o*f T^blfe 4.5 that most, if not all, of 
the posttesfs are related to more than one pretest, when predi(pt4ng any\ 
student ^s final achievei^ent relative to that of other persons in the* same 

classroom. This suggests that the u^e of only one pretest as. the cov^riate 

I . . • ^ ' ' - •. . ^ * 

fo?r*^a2lch posttest (i.e.,' the same test administered in the fall) is in'adequate; 

• '"^^ J 

it also suggest theoretical models which migift account for why some s%dents 
achieve more than others within the same classroom. For instance, it looks 
as th<Kigh the more literate students witl:)in any given class^om may get a ^ 

^head start on learning phoneme-grapheme correspondences. (Decoding '2). Moving 
to the upper section of Table 4.5, we can perhaps infeir that a western , - 
Eui^pean^- background as well as a prolonged* educat-ioo may also facilitate the * 

.acquisition of Decoding 2 skills, as does being a member of the male sex.. 
I A' similar ajjprbach to tt)6orizing about within-Jclas^roqm variation in . ' , 
achievement "cah be indulged in on the basis of the coefficients in, any column 
of Tal)le 4.5. • It could be argued that the acquisi'tion* of literacy might well 
Ke facilitated by prior word meaning .(Decoding having prior English 

.training in 'the United States as opposed to abroad, a, higher lev^l of prior 
employment", .youth, and masculinity (although we have no direr^ evidence to bea 

i ^ ' ] . - - . . ' ' ' - ^ . 

on. this, the last two effects may well be due tp the tendency for housewives 

to be isolated from English-laitguage stimulation, in t-he home, and 'for^dlder ^ 

individuals to be -assimilated into 'an^ ethnic cpmmunity). ^ ^ 

.* Achievement on the Decoding 1 posttest, relativ.e to peers ia^ t'Ke same ^ 

• ' ' ' '* * . ^ j ' ! * * 

class):oom, seems to be primarily a function of a ^t\ident's initial coipaand 

• • • . * ' / . - r * ' . 

of) the ^vocabulary skills required by. that test. It is noteworthy, however, 
that prior end current levels of employment may be. positive predictojrs while^ 



* • ^ • - t , ' 

the^ amount of English studied in the previous couafry is a negative predictor;- 

it is tempting to speculate that proper familiarity with word meaning must be 

acquired in a conte-xt of active use* ' , 

As for t"h^ prediction of wifehin-tclassrodm variation in achievement on the* ,* 

Oral Proficiency^ Test scores, we see from Table 4.5 that all three scoring 

systems relate positively to prior John Test performance. It i»s^ noteworthy, 

however, ^that> achievement qf proficiency under the more stringent scoring 

systems seems to be facilitated by initial literacy * Perhaps those students 

who have the abilfty to read better than their peers within the same^classroom- 

can g^in more experience witlj^the structural features of the English language 

through th^course, of the year. Achievement, on the Oral Proficiency Comprehen&iou 

score, oh the afeher hand, seems to be facilitated by a student's initial knowledge"'*^ 

; ^ . . " ^ . : 

of vcfcabulary, as me^$ured by the Decoding 1 test* Perhaps an initial -^repex^toir 

6>f root words and' syllables can facilitate the acquisition of sufficienj^vocabulary 

for comprehsnsion; noti^ ^that it seems to have little impact upon -the achievement 

of structural accurae^y,' however. ^ • - \- ' 

The pooled within-classrooms regressions of Oral Proficiency scopes on . . • 

^tbdent background pharacteristics seen in Table 4«5 are intriguing. It is 

■ :■• ' ■ , ■ ' < ' ■•• > .: 

apparent thac tiiOse 5tuaents within a given classroom who have relat^ively le^si 

' , • • . ■ . ' , ; • ' ' 

prior tr'ainii^g in Engiish, whether herij or abroad, are ,the very students who, ■ ' 
in terms bf final a'ehievement?, profit the mo^fc frt>m instruction.- A, iSimilar 

^ ' . j . . V ' k> ^ ' ' ' ' ' . . 

phenomenqha/ &e,ems ta , occur ^with respect to the variable **lengch of time in the ^ 

. /* , * ' ^' ^' • ' ' ' ' ' . ' * ' ' > 

United ^^1:a(tes»" Recent atrival^^who have had. little prior opportunity to live - 

"in an \English-language milieu make substantial .progress in. achieving; proficie^icy 

by the end ot the School year^ . ' * * ' . ' • 

, , V ; 
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, BETWEEN CLASSROOMS RELATI'QNSHIPS AMONG STUDENT BACKGROUND 4^ ^ 
'\ ' CHARACTERIS:riCS, ACHIEVEMENT, AND. TEACHER/ STUDENT INTERACTION VARXABLES 

V A, comparison of the total-sample regressions of posttest scores on student 
background characteristics* in Talkie 4.1 with the corresponding pooled within- \ 
* classrooms values i# Table 4,5 reveals substantial change in tKe predictive role 

♦ ) ' p > • • 

played by age as well as history^ of employment in the country of origin. This 
suggests that classrooms are differentiated not only by level* of proficiency, 
as measured by initial John Test scor.es, but by certain student background 
characteristics as well. In order to explore this possibility we turned to 
univariate ^alysis of variance and c'anonical discriminant function analysis, 
just as was dojie ±L Chapter Three- to study classroom differences in studentA 
teacher int-eraction patterns. ' «• 

Tal5le 4.6 summarize^ the^outcoiae c3f univariate analysea^C^^griance ^among 



I day-school 'classrooms on s^tudent background characterigjtics^^Hbtice £Ka.t Virry 

. • ■ ■ > r ■ ' • 

the amount of Englisb^studi^d JLn , the .United 3j:^te§^^3xiBs_KidsJj among" classrooms 

(not' surprisingly, since 'this ds , the be'st't^^^J^^^s^^^^rSsst scores recorcfed. ;^ 

- • ' "■•^'"f - r-— : ,: ^ 

• in lable 4.1).^ There is some indication, , howeyer, that previatis employment 

' •■ ^ i'/ ' J . ' ■■ 

and eduo^^onal level al30 dif|&r^f rora classroom-tcrclassroom, so it.is^worth- ^ 
while to study the outxrome of canonical" di^criminaiSit xuact ion, analysis in 
hopes of clardfying/these reiatJ.ons.hips. , • , . ^ - ... 

^ Not too surpriisingly^, one highly s^gnijficant discriminant, axis emerged 
from the "analysis, and it a^^counted f^r 59 percent of the between-ciassrqom . 
variation. Details of_the ailal^^is need not bVpreseated heria; suf/ice, it to * 
say that amount o^ English 'studied in' the U. 'S., English studied in the prev^us 
epuntrry, length df -time in , the' U^^-^ years of education, yquth, -and previous 
occupa^iCff^l l^el .were:.57eighted the first discri)ninant fupctijn in the* / ' 
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TABLE 4.6 



Univariate Analysis of Variance Tests for 
cfassroom Differences in Background Characteri^stics 



Day-Sohool 
F-Ratio 



Sex (F) > 
' * Age 
Time in U; S. 
Formet Job 
Job in U. S. 
C^Qrigia tNC). 



'Years Education 
Hr- S*- 'Diploma 
-Farfiier^Enqgr^ - 



t-. 



Between Classrooms Degrees of Freedom 
Within Classroom Degreds of Freedom 




V • > 



4 . VI . , 

- . . ; " 

■ . •■ ■ , ■ ■ . . •■ ■ • : . 

order listed. By taking into account, the second canonical axis, the percentage 
of variance Accounted ior could be brought up' to 73 percent, and the resulting 
solution is plotted in Figure 4.6, The second axis is of marginal statistical 
significance; it is a weightejl linear combination of sex, English studied in 

# • ' p • • • ■ , ■ " . ^ ■ • * ' ' " / ' 

' the previous country, aad national origiiv, in that order. * 

^' ' " ^ • ' ^ 

^\As for other .figures relating to discriminant funct;;fcor^ analysis, we have 

represented the classroom centuoids in Figure ?. 6 with unique letter-number 

codes,, while the student background characteristics have been represented by 

vectors. Inspection of the figure reveals wide' variation among day-schooX>4,v 
^ . / - * « % * 

» classrooms in'^terms of student background characteristics. It can also be 
seen that much of .the variation among ^la^rooms ig related to* .level differences 
as. determined by Johr^.. Test scores. _ ■ 



As* far as the background characteristic vectors plotted' in Figure 4.6 ^ • 
go^. there .seem to be some rather suggestive clustersju X^uth and ^piount o^ 
English^ S:tudiadJ.n the U. S,; educational and^occupat^ijral le^jel in the ^. 
previc 



w - - . _ 

;Lish^S:tudiad J-n the U. S,; educational and^occupat^ijral le^^l in the 

ivious country; amount oTEfngl^si1^rs^^^ in tFTe previous country and ^* 



- Occupational level in this cotSltry; sex (female), origin (Cuban), and length . , 

of time in^tha Uftited Si:at^s. All-in all, '"these results suggest the possibility 
' * th^t, students .have been assi^fcd .to more-or-less homogeneous ^classrooms on the 
/ .basis, of ba<|kgronnd characteristics' in addition to their John Test scores. 
/ ,It is' enlightening, to compare Figure 4*6 to the earlier f igur^es* In 'this . * ^ 

K''^ chapter^ in which pre- and-posttest ^hieve'men£ ^fe, plotted xdLthout taking 

4ny notice of. student backgrouncT'characteristics within the classrooms 
' .. ' fnVoived/ This comparison- makes the exceptional Literacy gains of students 

^ * . I ' ^ ^ 

' ip^classrpom CI seem all the more* imprdssiv^, for instance (Figure 4.3), 



, ' I' * 
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Plot of Day-School ^Classroom C/tntroidfi and^Student Background Characteristics 
in the Space Depicted byfthe Two Largest Cangnical Digcritninant Axe% 

i 
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When Figure 4*6 fs compared to Figure 3.2~a siihilar' analysis of 

.■ - ■ - ' . • ••' 

classroofn differencea^, but in term^pf teachfer /and student interaction 
^patterns — ^^several fascinating points of agreement emerge.' Notice that the 



^ two^^^cla^rooms tauglft by' t^ac^gC ark-widely discr^p^o^ with regard to 
student l>ackgrounds»qharacteris^|^M& (Figure 4. 6^, and * that tl)ese pvjo class- 
rooms mre^ handled in* widely ^ifflrent ways by *te<tcher C (Figure 3.2). These 
^ -^ata suggest that the mannej:. in. which .a part^gular plafes is taught 'may reflect 



a special adaptation on ths part of the teacher to theJ)ackgr6und characteristics 
.and ability .level eft the, stuJeJtts^J^olved, Although a bit,iiiore speculative 



than pur .2?e$tsoning with * regard tp teacher^s^i^^a^ that; »'th"e three 



[ tp teacher cr's" 

level i classrooms ]ta"^ght^J)y^teach'^r F^are^^r^^J^idely^i^ to ba^k- . 



7" - ground char act^risii'cs '(Figure 4^.6) and tbat*the'iiF re^^iiyej 

^ Figure^ 3.2 suggest, the applicatiQiv af c;^jnore directive Reaching strategy, in 

the iess^^well^^r^pared classrooms*^ As fot, teachers £ .afl^ t), there^seems to be . 

j a possible aj^sociat ion* between the_amount*.o£ stu3ent-student: fe^bacjc,in 

Spanish wh'icht occurs in a classroom (Figure •3. 2) and its"^ imposition, .terms . , 

\ of country of^ origin, eex, and l^gth .of. ti^e in the^lf.. a. (verti^l dlrectiori ^ 

■in Fi^ure'^^'B) ; a . similar Inf ef ence' c^n be dr.a*n for teacher C. \ 

/'X In sunmaW, we'hsVfe accumul^ated a' good dealVf J^idence sugges%ve pf * 

, relationships* between •clas'sropm inte;;action •vartSbles aj^student achi^evemeut ^ \ 

variable^. ' .T*he , regress ion anatyses^presented in earlier .^ections of this ^ 

I ^'riwpt^er^. were designed to reveal. jiQW-classroom intersactioiT experiences might come 

1'- to -affect ^tudent perfo^ance. Now we see J howevet, that student peorf omiance 

:^^n4 ba,(;l5g5P^uTvl. cHaracteri^ we^lt affect classropm interaction; i.e., 

*T ' ^ ' * • . ' ' j • • •• . ► * " . r ■> 

7> -BjaodeiEiof reciprocal caiisatiorn is ppobably laost appropriate for this dat 



locations in 



. In order to carry out a more systera'atic exploration of pxissible - 
relationships between classroom composition and the teacherjtstudent inter- 
action which t^k^s place therein^ we turneS to the technique of "^canonical 
correlation. -^ince the. primary product of ^ the canonical discrijninant function, 
analysis carried out in Chapter Three was .a sex of classroom centroid scopes 
on discriminant axes- (as plotted in Figure 3.2), and a corresponding* set of 
ciassroom cet^roid scoffs is now available with respiect to^s^fudent background 
chaxacterisitics (as plotted* in Figt^re 4.6), we can apply canonica!^ correlation 
analysis to, seWch for systemat^.c relationships between theae two sets of , 
classroom , coordinates. , In fact, candhical correlation analysis seeks to fin 
those orthogonal linear combinations of the scores in each set which are, in 
successive order, maximally related to one another*' ' '\ * ^ ^ - 

^ The potential benefit of applVisS canot^ieal correlation analysis to the 
classroom centroid coord^iaates" derived* 'sep'arately througfTcanotvic^l discrimiirant 

function, analys^is .of teacher/ stiideht interaction^^d stud^^nt fcckground 

> ^ ' , * . ^ ' . • ) . 

characteristics (or student performance -variables) stems from the fact that 

we- are dealii^g with highly^ reliable indices of between-classroom variation 

In each ca'se. Discriminant function analysis^ s^ryes to^aximze between- 

^ . * ' • ^ ' I . . i 

classrooms variations relative to withinrclassr^oms*ya;;iatiop.. It hps 

already allowed us to. a&s^ss classroom differences relative to d^y~tb-day 

variation in. teacher/ studgSit interaction (Figure 3.2) as wel^l as* cla^ssrooip 

* ' -J* •' '"'^ 

* * ' * -iv. ^ ' ' * ' * \ 

differences relative to student-to-student variation, in background character- 

» " . ' • A 

% . ^ ^ \ . [ , ' 

is tics (Figure 4*. 6); these aipialyses were dond separatel^r^ however. ' ^ -,^s 
Whether dealing with'teacher/studefit interaction or student characteristics; 

the;panonical discr.iminant function arialyses yield rell:able^"±l\Tl±ces of variation 

\ ^ / ' ' . ' * ' ' ' , i 

J. ^ * . * , • • 

'at the diassroom^level; the pcfssibillLy o^ relationsli4^s app^g thfei^e dif ^fetent 



doinains of variation at the classroom level can thus fee expldfed through the 

,use of cancffiical correlation analysis of classfoom. <;entroid projections onto ' 

/■ . ' / ; 

the discr'tminant function axes. • * *- - ^ ^ 

^ : - ^ ^ i — ^ ' ^ ^ Mb \ 

Since our interest in variation between classroom^ inclufies n^K)nly 

tea.cher/stud^n^' interaction and student background characteristics, but 

tudent pretest^ and posttest performance 'as well, the' letter variables were 

included in a new canonical discriminant function analysis comparable ^to that 

conducted ih connection vith Figure 4.6# It is. not surprising that multivariat 
^ ' \ •* » . 

analyses of variance on pretest and pc^t£ e ^ st acl j^evement ineasures indicated 

, ^. ' r * ' ^ ' 

'hrighly significant, differences «Biong the 14 classrooms in the day--seho0i-~-&ample 
for everything e/cept Decoding 2 scores. These level 'of achievement difference 
were ire^f^.ected in thC^ coiiippsitioti of the^.canonic&l discriminant axes derived 
' from^studerit background, ' pretest and posttest variables combined'. ^ 

)• • ■ *• • - ' ■• ' ' . • 

The first two /canonical axes accounted for 84 percent of between groups 
variation, relati^^^'to within :gr6iips variation, and -weighted primarily the 
fall John Test and fe^px.ing 'literacy Tese,^respecti^ely. In ^all, fpur sigrilfi- . 
cant canonical disc^i^na^Jt Junction axes were obtained, accounting for .92' 
pe^tcent of the 'relative;:v?5ra^'an?ce araon^- classrooms. We will not report the 
analysis in more detail ]\ef^ however, *sin<!e it set^?& otily to provide 
classroon^cefitroid coordijia'^feS for iiiput • to canonical correlatioti. analyses. J 
As input xo "the cananifcaL corJr'-^lation analysis of classroom means on ^ 
canonical, discriminant func.ttcji axes, we, used the four composite ^cores for 

Student characteristics lust mentioned ^-s well^'as scores orr'the first five 

^. • • • ' • - , 

canonical discriminant function axes derived from classroom interaction data 

(Table 3.6). '^'ive cla*gsrodm' interaction canonical scores were used insteail 



I 




of only the four emphasized in other sections of this report bfe:&use o^ ^ 
evidence- we foAnd while ^coTfdT^ctinf^, regression ^analyses (repo rted earlier 
in this Chapter} that reliabii classroom interac7ijpn differences o^yilst^dll^^^^r^ 



beydnd the oi'igina^v four transfgrmed classroom interaction contras-tsv Nptice 

( ' ' . \^ J - ^ ' ^ ' 

that the fifth c^rioniSdiOi^is repotted in Table S.*6 is marginally significant, 
in-line with this finding, . y 

Th^^canj^nica^^ from mutual prediction,- on a'! 

.^classroom-by-clas^room 'basis, of'stiftVent baclcground and achievement data from 
'-classroojn' interaction data Jor' ^^ice versay.we^.e ,96,^.7^, -56, and ^^i] Everf^. 
allowing, for a sample* size^^-^ly^lA^Iii^SlU A criterion. y^eided^^a-^^niiicant _ ^ 
value o£ 33.7^with degr^ of frepdem, c/onfirming o;4r suspicion-^that 



what goes on in classrooms with respect stable pattern^^of teathe^^student 



inter^tion iav highly relate^ 'to studen*t-t)ac'k8round.,and performance ^ract^stics. 

• , ^ ■•.Pa.-** 



The Structure of Between Classroom 
Varlation(iri the Adult Learniri^ tJlntpj ^ 



The canonical coj^relations reported above give evidence of rather stfbng 
relationships between student b^citcTround/perfor characteristi^fed 'and 

I / , \ . ' K • * / ■ 1 « 

teachear/student interaction' factorfe att-^tiie'claasroora level, when dealing with X 

■■I ' ■ ' ^ ■ . ' ^ i ' . - 'C ' • 

Scoresj' on canonical discriminanf function axes. It is in ard'er to asjk whether ' 

we might be able to relate these relationships back- dovm to the level of 
' ^ } ' . ' * - ' ' . ' . '-^^ - [ 

indjtvi.4ual background characteristics, test .scdte,s, and classroom inj:^raction 

factors. The answer is yes, although we must keep in mind that the tesults 

are dependent lipon only 14 classrooms studied in the Adult Learning, Center 

' ' " ; ' ■ . , " ' ^ ■ ' ■ r "■' ' 

and may not generalize widely. Only where a similar system of assi.'gnment of •. 
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students to cXass rooms ,^v^^similar ranfi"in teaching styles, and a similar 
"population* of students gan be ossMied tp exist wili>thes.e results generalize 



I to other ESL systems. ^ >^ ^; 




Table 4.7 contains the fip^l .outcome of a serig^of projection and'' 

^ f . » •' . ^, n"- • -f" 

fctansfonnation. operations , In brief, the two setajof canonical correlation 

^ ^'weights derived from relating student bacjjcground/perf orin^ce ,c.las*sroom tientro^^ 



s 



to' te'acher/sttident int:^i;^ction classroom c^ntrofds we/e used .Co^ trans forjn' 



betwefin-classrooms projections^ of individual sttfaent ba^ckground characteristics, 
^test 3core,s, and, clas'^room interactidn factors 'SO as to relate to a cOramdn ^ ^ » 
canonical orientation Af^^es^vThfen the 'resulting combined ipatftix of projections 

^' t • • A" ■ ' .i- T- ' 

of or'iginaDL*varic^bles ojito comracwtsaxes (subj^^ct to' some uncertainty dUe to 
; less-th^n-perfect 'canonical correlaitkons) was transformed in an att^ijo 



. Achieve a positive manifold simple .structure, usi/^g a version of-* DIRECT GEOMIN 

rotation (TabJLe 4.7),, . * \ ' ' \ ' ^ 



the relationships between strident background/performance variables and 

. ' ■ * ' !' ■ 

teacher/ student interaction variables fot the 14 day-school tlassrooms represented 

by all four c^anonical correl^'ation^^ (♦96f^ .74, .56, .34) were included in^what 

^ , ^\ - ' ' " ^ ' ' 

, , led up to the results presented in Table '4.7, even though the last two canonical 

. .correlatioiis are quite low in vAliie. This v;as doiic because wp "found that 

■ - , ; - - • , ■ " 

tation of all four common axes £or posftive manifold simplia structure in 



ro 



1 their between-clas-srooms i;elation^'hips t© the* original vaSc^^es did' not yield 
a marked change dn^bs it ion from the original canonical *||^:ientation; therefor^, 

^ afiy uncertainty in the relat.ionsh|Lp between student background/perf-ormance 

^ ^ — • ' ' ^ * V ' . ' ' • 

variables and teacher/student inter'actlon factors due to imperfect canonical. 

■s ' ' . ■■.../ -r^' ^' 

. correlations is largely confined to the later "factors" listed in Table 4..7^i , 

. ■ ■ ■ ■ • . •;■ : ■ . •• ,\ 
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Between-Classrooms' Factors of Student Backgrou]nd Char^ct^risticfi, Pre- and 
,PostteSt Achievement^ and Teacher/ Student Interaction; Adult Learni'ng'**€enter' 



Background ^ 

' ' * Sex (F) 

* ^ Age» 
' ; Tiiae.ln U. S. . 

, I ♦ Former Job » 
\ ! Job in ;U. S. [ 
& I , C-Origin (NC)i \ 
\ Yekrs ^Education. 
- I ' H.. ffr Diploma 
\ Former Eng. ; ; ^ 

• i ' En^. in U. S.j 



.00 

-.04 
.38 
-.21 

'.04' 

- f 

.•22- 
.3&- 

■••It 



Factor Pattern Coe£ficients 
" - . •• / ■ 

• II' III 



if 



.03 

131, 
i .05 
•36 
' ;05 
.0,6 
• -.17 
••24 
, .04. - 



i"-.^.74,. , 
.'03 - . 

•li . 

-:34r- 
,, ,^56 ■ . 
■ •AI 

.14 



■ • IV 

-.21 
^ -•1-6 
^ -.75' 
.;00^ 
-.02 ! 
' •45- 
^.25'- 

- -•'21 
. -.JLI ■ 
«. ■ ' .21 . 



* f jretest 



D2-F 

L-F 

H-F . 



' / InCe'raq^tlon, 

' QA-IND ,0 ' •> 

' * CFrP-A . . A 

\ SFBK • ; . 

1 FREE ■ [- , 

1 "OTHER V. .i- 

aiR READ 

; MOD^PRAC . 
*1 'iSFr-M-P V. . ; 

' • jQA vs. jym ■ 



Posttest 




63 



.'59: ; 

.66 
.52 
..35 ' 



-.09 
' ■;.-.04 
•-.73 
.57 
f-.03 



I 



V 



'. 10 

.35, ■ , 
.50 ■ ^ 

•AI- 
.35 



•23 • 
.0^ . 
. 15 \ 




' -•6i; „ 

.05 ^ 

•21 ' 

•^-.32-. 



r •E 

1 •IS 

i-; -'.50 



.02. ^ J I -.18 

?^«s.28 ■ " 1,^, -.22 
Ii42 . " x^-l .-i05 



.10 
-.41 

.io 



1.00 , 
,. .20 

. ;'34. - 



.35 



, ,-.23/ 
-.51? 
-.15;, 
'.O.'04j 

.23i>«»' 

-.11 
r.44 

,-.55. 



\64 

4" 

.01 
.51 




Factor^ Intereorrelat^^ons ' 



' M.ob :/ 
.21 . 



9^, 
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' Interpretpjtion of Between-Classrpcxms ''Factors" 
of Student Background Characteti'&tics , Pre- and 
'Posttept Achievement, and TeacheV/Student Interaction 



- * .The "faptirs" given in Table' ^.7 can be interpreted in much the' same . 
way as those resulting fro,m conventipnal factor analysis as long as it is 
kept* in mind that we are discussing hypothetical determi^iants of manifest 
variation among the 14 classrooms at the Adult Learning Center. Although only 
14. classrooms were studied, we know that the classroom centrqid scores which 
served as the basis .for this analysis were highly reliable to start with 



.e. , th^ 



showed maximum variation amorg classrooms relative to variation 
within classrooms—whether ^y-to-day for teacner/studei\t. interaction or 
student-to-student for backgr^uad^ characteristics and achievement) . 

Notice that the background, pretest, interaction, *and posttest variables 
listed in Table 4.7 have been organized and grouped in the temporal sequence 
in which they occur in practice. We are not attempting to infer how these 
domains of variables 'might influence one another, however — that was^ ,th€f role 
of the ^regression analyses reported , earlier. Rather; .we are attempting here 
to discoiJ^er what factors might ^determine' how students from, various backgrounds 
and of* different final as well as initial ability levera might come to^be 

• '. • • , •• ' •. ' - . ■ f ' / ^ 

exposed "to .var 4.0 us specific .forms of student/teacher interaction within the 

>^ ' - ■ * ■ ^ * ^ ' 

14* classrooms of :the Adult* Learning Center. 

In attempting, to inifer the bases for claasroom-to-classi:oom variation 
'in, the Adult Learning Center through inspe^ction of the coefficients in 

-'.'.■-'1 •/ .' ' . . ' ' . ' 

Table 4.7, the first, thing we can note is that most of the variables h^Ve 
like-signed loadings across all four factors and that the factors show generally 



ERJC 



positive Intercorrelatlons. • These feature^s, cpjupled with the fact that the 
John and >Iorano pretests as well .as the OraJL Proficiency correctness posttest 
score are positi^jely loaded on all four ,^a :tors, suggests strongly that all 



J. 



of the factors are getting at different aspects of the same general distinction 



airtong classrooms within the Adult Learning ICenter; level of Englisfi language 
proficiency. ; ^ I / 

? .4 :/ '■ ^^'^ . -. - • 

Each factor caji therefore be regarded^s. a specific bipolar contrast 
between high and low level clas^^oms.^ Ij^^ interpret the loadings a« they 
are presented, we will thus arrive, at ^d^^^tions of h)rpothetical ways in 
which classrooms might come to contaip' stjMints of superior proficiency in 
English (withi^n the Adult Learning Center)]^ r;eflectipn of the signs of the_ 
loadings, can therefore indicate how AdiiW^^^rning Center classrooms might 
come to contain students of low,er profitil^cy in English. ^ 

'^^M ■ ' ■ •' V 

Looking at the positive pol^ of the igLgst factor, we see the likely tendency 

• / ^ ^^ : ' • ' " 

tot superior Adult Ledfrning Center classrJaias to contain students who have/a 



history of higher^than average English Stgdy; in the U. S. , Level of Former 



Job, and Educational Level; while having Hoyer than average Age ^d level of 

Job in the/l/. S. Moving to the loadings of; pretest performance on t^s factor, % 



of competence are getting at the s^me 



we see a> tendency, toward superior perf orii%i^ce on all initial achievement ineasures 
It is/ not surprising that these indicators 
aspect of variation among cla&srooms as^^t^or Englisli study, level of former job, 
'and education. The negative loading foiflevel of emp,loyment in 'the IT. is 

' ' . ' 'Vi^ f • ' ; • - • 1 . " 



interesting, however, and may betray, tlie fact that stu4ents who have the 
/[ time to engage in extensive day-school English-language training in this 



couittry- tend either to be unemployed or' underemployed, compared to their • 
former job and educational level (hence t^e motivation to improve their 
; English) , ' ' • • ^ . ' , 

The loadings of student/teacher interaction variables on the first 
between classroom factor imply above average prevalence of the "free 
response" mode of interaction,- as m£ght seem ap^propriate for students of 
higher than average initial ability. * Note the tendency for student-student 
feedback" either nqt to occur or not to be allowed, along with the "model- 
t)ractice;-corrective feedback-model-practice" paradigm. There is a tendency 



for students within iplassrpoms with such a high level bf prior training and 

■'.>)' ' ^' /' ' 

initial ability not. to-be expected to read and/or ask qtiestlpns undfer direction 
of the teacher; 6n the contrary, a (group?) question- answer int^e^^action pattern 



^ goes aiong with "free response" format of instruction; . / / ^ 

.* *■ •• «' 

As for the final achievement implied by this first factor of between- 
* ^ " ' "* 

classrooms variatipn, -somewhat above average Oral Proficiency correctness 

is • indicated along with below average Decoding 2 pe/rformance. Here we see 

•again the phenomenon noted in connection with Figure 4.2; the ^implication 

. " • 

is that students iiji higher-level classrooms do not get exposed to instruction 
which deals with pronunciation in terms of phoneme-grapheme ^correspondence-^ 
they consistently end up being surpassed in these skills by students in the 
initially lower-level classefe# The latter clasa3K)oms define the negative 
of the first factor.; notice that students »there have higher than average 

■ ' •■ . ' ' ' " 

/^of employbent- fh this country, and are 'older. • ■ 
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At the positive ^ole of the second factor of between-classrooms , 
variation there is revealed a tendency for some superior classrooms to 



contain students' who have ,been in this couatry for a relatively long while 
and who are established in liigher l^vel jobs, ^his goesVlong with a history 
of English study in the former country and backs up dur earlier conjecture 
that prior English languiagk training may, bfe ^^strumental in'' the acquisition 

of higher level employmenti in thi3 country. Note, however, .that ^ high 

I , I ' • ^ . ^ • 

' * I / • ^ 

overall level of* education ig nbt^iridicated. 

All of tjie pretest' alchievemejit scores except Decoding 1; the vocaijulary 

^ indicator, are loaded on tl^is /factor. Glancing ahead to the loadings of 

. ' / ' ■ * . " 

posttest scores., we find- tha^t we are dealing here with a factor which relates 

/ : i t / / >7 * — ■ - - 

t^eMj^tipnal acljieveDieiit of •posttest literacy, along with the above-average 



achievement" of prdf Ipien 



interaction facto 



"7 



is 



/ 

1,1' 

ly .characteristic of all four factors. 



When ±}: o^mes to fha/nUne classroom interaction factor loadings. It is - 
not 'surprising that the f )iirect-read ai»d/or*ask, questions" pole of thfe last 



tj^ljghly adsociatied 



relating. to posttest literacy. Other, 
patterns are "OtJi4r"./aiid "Free," aid 
f eedback-prompt-ai swier^' sequer 
"Othe^r'T activities which ootfUr with hii 
.Adult -teaming 'center classrooms 

Most of the/ student , background ch^ 
of bet</een-clas6itooms variation 

the positive pb$^ as indicating 

r Vjl . " ' 
yell employed females who have bee 




is^'between-classrooms factor 
r is tic classroom Interaction - 

\ • ■ ■ V 

"ques'tion-answerrcotrective 
U3 to ]|uspect tl\iat^some of th^ 
equency within^ certain higher-J.eyel 
nvoive reading , materials . 

eristics. load on the^ third fact 
Learning C;en£er# We could yiew' 
cy for superioii classrobma to ccintairi 
/fcively Well educated. Individuals 

• ' - Pi '■ ■ - 'R.. 




rj 
I 




in thes6- classrooms^ who may well have arrived from Cuba some time ago 

(relatively speaking) , have had superior English language training in their 

country o'f origin,' but not necessarily in the United States. Age is the 

only other background variable' not related to this factor, or . ^ ' 

^ It lis interesting that *the Morano pretest has a lai^e loading on,^he 

^ tl{irT^between--cla^srooms factor, especially given, its minor rel^tiahship 

to other measures of within-classrooms variation (Table 4.5). The Morano 

score is an indicator of grammatical skill. Looking ahead tp the loadings 

of posttest measures on the third between-classrooms factor, we see that 

♦ 

the oral proficiency correctness score. has its highest loading here. Taken 
.together with the high loadings for posttest decoding^skills it appears that 
substantial ESL achievement tends^to occur in Adult Learning Center classrooms 
characterized by the positive pole'of the third factor. This could well be 
attributed to the hi^h current as wfell as prior level of emplbymQnt^' farmer 
ESL training^. educational level, or sex (female) of the* individuals involved. 
Notice that only the'^Other" factor of classroom interaction has a position 

loading oc the third factor in Table 4.7, and the "teacher-direct-student 

/ « » •. . . • '» 

/ ^ 

read and/or ask/questeion" pole of the last classroom interaction factor 
predominates. / Other classroom' interaction patterns are deemphasized, Including 
"model-practice" as well as "questioh-answer-corrective f eedback'-prompt-answef • 
This l6avds us xd.thst little basis for speculating about what does ^o on ij^ these^ 
classrooms; perhaps the student background characteristics mentioned make , ,^ 
instruci^^ii, ^irtelevant to achievement. ^ • • / 

Just as the third be twe en-classrooms factor is strongly^ relate4' to^ student 
TjacJcground characteristics, the fourth factor is strongly re-lated to all but' 
dne of the original nine classroom .interaction factors*. The pattern of 



loadings suggests the familiar contrast between ^claBsroom interaction^ ' ^ 
factors seen aioftg the horizontal axis of Figure 3.2: QA-IND, CF-P-A,, 
SFBK, FREE VS. MOD-PRAC, CF-M-P, DIR READ, and OTHER. At. earl'iei^ point4 in 

^ ^. : . ' . ' ^ . - 

.our discussion we havC referred* to ,th^ positive (left in Figure '3.2) pole 
*as flexible, free, supportive^ eclec*tic^ individualizeci, while^ referring 
to the negative pole as structured','^ lass or/ group oriented, etc. 

' Those ^lassrooms in which a nlore" supportive and individualized pattern 
of teacher-student interaction prevails tend to contain students, who are 
recent arrivals frim^Westem Europe, .perhaps not too highly educated , a bit 
younger than average, and male. This means^ of course, that the more highly, 
structur^4g^classrooms (e.g., thcTse .of teacher F in Figure 3.2) tend to contain 
older females 6i Cuban or Carribe^ origin who have been in the United States 

s 

for some time. As for the t^est performance, it is clear that the recent* 
arrivals- fjom Western Europe are very deficient in Decoding 2 skills (phoneme- 
graph^e correspondences) In the fall but become "exceptionally proficient in 

4 pronunciation by the end of the school^ year. Again, this, illuminates the 

/ ^ --^ 

/ • - 

tendency uoHed in connection 'with Figure 4.2 for Decoding 2 scotes to- increase 
, dramatically In those classrooms d!h which they ate initially low, and not 
to cUange ±f/ they are initially high. / ^* ' 

I What these four factors of betwe en-classrooms variation" signify in a 
^global s^se is th^^ complexity and richness of the ESL training -situation 
ih terms of student background characteristics, initial i^'st p^efformance, ^ 
student , teacher Interaction patterns, and* final achievement. The Adult 

^ ^Learning Cerite^Wst be viewfed as djmamic system in which students are, 

• ' • ^ * ^ . • ' • ' ' , 

^- ci^aimeled into c-Jassrooms which prbmi^*to^provide ^them with an optimal • 

^ ,i Learxiiilg experierifce._ Table 4.7 indicates that there are at le'^st four ^ 



ways in which classrooms cai\ come 'to be above average in the achievement of 

English language proficiency at the- end of the period of instruction. It ' . 

is. clear that initial proficiency as determined by the John Test is a- good 

« * • * • ** 

across-the-board guarantee of^final proficiency, not surprisingly;' but> 

among other aspects of pretest performance ^ background characteristics'aud 

classroomt interaction patterns must be taken into account in of^der to fully 

characterize the variety of patterns o-f achievement seen in different classrooms 

In discussing Table' 4.7 we have mainly takep note of loadings which 

exceed .20 in value. Since our discussion has necessarily been brie£^ it 



youldbe useful for * the reader to carry out ^ detailed 'ihspection of the 
entries* In TabJLe 4.7 on their own. This analysis was arrived at through a 
rather unconyenti^onal approach but it shows promise as a wky^to. ^Mate.between- 
tlas^rooms variation ijj^student characteristics to between-classroom variation 
Vin daily patterns of teapher/student interaction, without the need' to r'ecord 
that interaction on a stu5ent-by-student basis. 
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INTRODUCTION. - * i . . . 



In this. chapter we present essentially tlie same information as in fhe 
.preceding chapters but in a less technical way. We alsa^discuss th^ alternative 
interpretations of ^he data, .the ktnd§ of hypotheses about teaching' that ought 
to be considered, and some proposals for action that derive »fropi interpreting 
the results of this ^tudy. The goal is to make the technical, information , 
presented in the preceding chapters. available to a larger audience and in " 

pai;ticular to the sfaff of the West New YorkfAdult Learning Center and to 

. ^ .... . ^ ^. .^^ , , _ 

member^ of the State Department- of Education who :may Wish to present it to 
tUeir coll^agjies who^have less ted^nical training than is required to read 
Chapters TnVee and four. ' » 

The preceding ^chapters presented a wealth of factual informatioti: about 
the characteristics of ^ the stydent3 in the Adult 'Jieaming Center in West- New 
York; a detailec^ description of t*he teaching styles of the teachers; a ^escripti 
of they/tudents^ Wchievemen t in acq uiring, prof iciency In speaking Epglish^ and 
-an analysis of the relationship between phe nature of the teaching styles 
employ^4 by the teachers, student pharact^xistics and student achievement, - 
These chapters Jiave been presented dellFeirately in a technical style. They 
represent a set of facts and the results of analyses which a ^reader may examine 



to see if lie or she would draw the same conclusions from the information that 
has been provided. . * ^ ^ — • ' * • 



' 'Thi^ chapter is more than a summary. It is a briefer description of . 
the work of the project and tJhe~ results of the research methods that were 

employed. It also provides inter^iretations of these results with a view to 

^ — ^ — ^ — 1_ — .. * • « <« * • . * ^ 

suggesting courses of action* * * • * • ' 



We remind * tWie reader, as have at frequent points in this document, 
that the results of this study .are not generalizable to other samples of 
Students or teachers or ad^plt learning centers except insofar as thfese other 

'^studealTfe or teachers or centers, have characteristics highly similar to those 
of the West^Nev York Adult Learning Center. Also the methods of the ^tudy ^ 
do not, strictly* speaking, permit us to dr^w stribt causal conclusions; we- 
can only* make inferences about probable causes. . . J- . 

^ The*practitioner, however^ is interested in knowing what might be done 

wl^th the results of* this study^ "Tn the latter sections of this cTfepte^j^e 

will sake suggestions ^^about such courses of action. Our presentation there / 

will 'consist in presenting the results of the study as premises in a rational 

analysis of practical action. • In so doing, wa will suggest -the implications 

of these 'premises .ftr practical action, and inevitably will present different 

courses, of action. ♦ ^ . * # , • 

' Since we have, given ample warning, the reader will re^cognize that we 

ai:e not presenting t^e* results of this study as' definitive statements about 

effective teaching of English as a second language. With ttiese warnings in 

* * 

mind, yhos^ origins^ are' in ttie logic, of statistical and empirical analysis, , 
we ^can suggest ways of thinlcing aljout these data that would lead to practical 
act!^on» ^ " I ^* . . " • ; • 

Others fiiay^ develop their own plans for practical action by following a 
similar kind of reasoning.. Those who undertake practical .action based upon 
the mnalyses that we have presented ought, also to- evaluate the tesults of 
these actions. If that course J.S fbllpwed", the conclusions drai^i for. action 
are tested, and their utility can be assessed. ^* 



•TU'e order c;f presentation of ideas and infdrmatiori in the fbllowing , 
"Section is: ^(1) the major questioas and problems to be studied; (2) thfe 
methodology of the research project; •<3) ac description of the students of * 
the Adult Learning Center; (4) the analysis of teaching performance; (5) the 
relation of teaching performance and other variables to the students! achieve- 
"ment of proficiency during ^e c;purse of the study; and (6) proposals, for ' 
practical action based upon the cdhdusions of the study. . . . * 

* . THE >1AJ0R' QUESTIONS AND METHODS OF THE RESEARCH ^PROJECT 

^ ^W^iat was the purpose of .this research project? The purpose was to find, 
if they existed, relations between teaching performances and the achievement * 
of proficiency in speaking English. ' The iilea w&s* to- cbll^'cfe information 

'about these perforiaiance3 by observing * teachers on* a regular bas^s who were ^ 

• ' y ' • • . • / ' 

teaching English as a second language to the students in the Adult liev^l^g ' • 

* .Center. The initial status of these students on measures of proficiency was * , 

/ to be assessed^ their proficiency at a later point in time was also to ba . 



assessed, ^he original methodology planned was to i^lafe the performances. 

or tne teachers^ tjp^ differences in acquired proficiency. * . j • * . 

' \ \ ' it . ^ ' • . ? • 

4 Several methoo9 *are •ava.ilftbi'e for*' achieving this goal. One method is 

^ to correlate -different, kinds^^bf teaching performances with. the finai achieve- 

> ment scores;"*partialltng out the initially status of students. The idea behind* 



*£liis method. is to* identify which particular performances are most Mgljly 
correlated with ^'nal, status. The statistical flf|thodology, that 6f partial' 
.correlational 'methods, utlliLsifes informatiqo^about^ln status. Ifsually 




the students* initial scores are highly correlated with tfiMr final scores, 

and this correlation needs be taken into account if one is to assess the 

* ■• 

unique relation of one or more teaching performances to final status, 

When this method is used, the partial correla»tions between teaching 
performances and final status ar/a produce'd. , Let us. assume, for examprle, 
there is a teaching, performance that, has been observed which wfe will jcall, , 
'Modeling (in this, performance t}ie teacSier proyidfes a verbal example of the 

* 

desired speaking performan9e) , and that a stat^-stically significant ct^rxelation 
of ♦SS is obtained 'between this teacHin^g performance knd a. measure Of profiqien 
in speaking English. This correlation of .35 (which is not an actual «finding, 

* . . . • . .v^ • • * . ^ — ^ 

but only an example) is the. association between Modeling .scores and the* acnieve- 
ment score when the* coi:relatdon beteween.the initial achievement score and the 



final achievement score has been controlled for tthrough statistical mean^^^^fiat ^ 
j.s, has beeh "ijartialled out."" The appropriate c<^nclusion^tc b^^rawn from / 



data of this kind can be reduced to the following ^t^t'ement : * the ^correlation 
between Modeling and achieved ^pjrof iciency in speaking English is .35 when the^ 



cdrrela,ti^qf between the st;u4ents' initial* score and final score has. been 
accounted for. \. ^ « • . ^ , • ^ 

This atatement is the literal empirical result. . What it means is that^ 
' - / * . ' m' \ * • . ' 

, we* can /predict differences in achievement from 'differences in the 'degree to- 

which. /the teachers use Modeling as part, of their repertoire of \.teaching pei;for- 

maiices. The magnitude of the dorirelatiori coefficient* gives us an idea of how 

good^ a predictor the teaching performance is^^ '* . * * ^ 

A t^ical wavof describing the ut^.lity of the predictor is in terms of,-^ 

tl/e amount of variance* in final scores 'accounted for, *br ^he amount -^of variance 



in^the fl^al scores that can be predicted from information about the predictor 
variable. One method used to' estimate the utility of this predipCor is 'to. 
square the correlation coefficient and transform the square to a pfere^age; - 
for example, a correlation*^ of 435 squared is .13, or 13 percent. An appropriate 
statemeijit is th;&t this, teaching performance, independent of the initial status 
of students^^^eticounts for 13 perdent of t^he Variance |Lj|^f3.Ti^l scores. ^ , 

The goal of .the research methoIdQlogy is to account for as much- variance 
in the outcome measures as pos^i>le. In a study of this kind jrhe* researcher 
uses the information that he has available on teaching performance, but has 
no idisa in advance how .many dftd which of the pre.dl-ctor variables are likely - 
to te^'^gboii" predictors (except when similar studies havie already been donfe). ^ ^ 
So the empiric^y fiie|t:hod^4s i*^ ^^ss^nce an inductive analysis of the predictors. 

of student" a^le^ement. - ^ " ^ . 

^ ' \ * , ■ V, - V - 

What tfiis means in. £ his particular instance ^is 'that a vatiable' such as 

Modeling is a. predictor; that* is, with a Significant . correlation coefficient 



0 



of .65,* this v§riatle accojints ,faXv I^JOut 13 percent of the^, differences In 
act)ieVem€;jit among Jtiii^ts at the end of the research perigdi"" Somfe^ other 
teaching perfCffJUSnce- or some other variable may account for more* or less of 
' the variance. ' * . , - • * ' 

The goal ±s to. measure the partial correlation of the. predictors * 
(teacher perf ormancjRs) and achievement (student proficiency scores) to see 
if any are statistically^ significant and to estimate how much of thp differences 
among the* students in final ^measured .proficiency is accounted ^or by each of 
these predictors. ' The result wdifld be a listing of the predictor variables 

' V % - ! ' . • "\ "^^^ . ■ ' - . 

that, were significant "and the amount of the variance fpru^hich. each accounts. * 
This ap^oadh is t3rpical in most educational Research' on teaching. • 
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. . . , • i& . 

A second approach is much like the first ^one except that, it uses the 

ba^ic information in a different way, ^Imagine a list of teaching performances • - 

V f 

such as nfodeling, eliciting, pr^ctice,/positive anvj negative reinforcement, 

. ""^ , • ' • 

and so forth. Ordinarily the researcher will find correlations amon>g these 

.various performances. Thpy^'lare .not truly independent,, and practical educators 

are aware of this concept as. evidenced by. the fact that, they frequently describe 

.teaching in terns of styles or dollections of performances ,whicli are characteristic 

of one teacher but not necessarily of another teaplt^r. From a statistical point 

- of view -Jthese intercprrelatUons ought not to J^e ignored. 

^^ultiple tegression -methods are available that takie into account. the 

.liintercorrelations among variables. When a multivariate analysis is performed, 

. ■ , V ■ * ' * 

the result is a multiple R,whicfi is a correlation coefficient. This correlation 

coefficient' is generated by entering all of th^. information available, for 

example, the information on the-v^fous kinds of teaching performances, into 

« • 

^ analysis which yield's a prediction equation . A predictiqn •equation takes . 

generally the form described below. We will us^ a simple example •cofistructed 

to illustrate what'is involved in multiple regfr^ssion analysis. ' ^ ^ 
Assume that we have four teaching performances: X^^ — modeling; — practice; 

X« — positive reinforcement;^ X, — negative reinforcement. These axs^ four descr;lp- 
'tors of teaching performance. For each of these variables we will have scores 

for each of' tHe teachers participating in theVtud^^ We correlate their scores 
.on each of^these measures with student achievement*and. ^fd.th each other. Assume 

that the correlation matrix looks like this ^note that these^are not actual 

correlations from this study) M ' « * * • ^ . 



* -v • ■ ... 

ERIC • ; 



• e 



, y -P • 1.00 .78 .so- ^-.34 .42 .10 

* J. ' ^ 4 ' ^ 

• 'l.OO - .35 '• .20 .45 -.35 

•• ' ' ' . 1.00 .30 ■ .32 .15 . - - . 

X_ ' • 1.00 .70 .^35 ■ • 

- X l.OO'" .15 ' ■ 

' X, • '■ ' . 1.00 

• 4 • " * ' ' c ^ ^ 

r,?^ .35, p<^ .05. ' 

, P_ : one measure of oi;al proficiency . - . ^ , 

J • second measure of oral proficiency ' ' — 

X^: modeling; X^: practice'; X^: ':ppsitive reinforcement; T ^ 

X. : hegative reinforcement ... 

We have chosen" to, use' two measures of -proficiency in this exiample, P^^ 
and The teaching perfon^ante variables, are those listed above. The ' 

numbers in the table are the zero-order corpelations * ([they are not the palrtial 
correlatd^ons) between the scores on each of variables. Look in the 

lipp^ left-hand , corner of the table and read across from P . We see that thja 
-correl'ation between the twp n^asures of proficierity, and P^, is .78» A 
correlation of this magnitude, is substantial and means'^that 1:he two measures 
..are highly related 'to each other. (We have also indicated in^the 'tablfe that 
correlations above #35 are statistically significant.) Again, reading across 

the^Jfirst row we see that the. correlation between P^^ and X^^ j(modeling), and^* 

♦ * ' ^ - • '■•>%'• - 
between P- and X (positive reinforcement) are statistically significant.. The* 

' - ; 1- 3 , : ; 

correlation between P and X (practice,) is very, close to si^ificanb,e, but. 
the correlation' between P^ and X, (negative reinforcement) is not significant. 



T^e way these data are interpreted/ li. .one is looking only at these M>i^elatlons, 
is that modeling and positive reinforcement are coi;related significantly with 
the fij'st measure of proficiency (P-)» \When we translate these-correlations 



i 



into the percentage of variance accounted for (by squaring the correlation 
coefficients), we would say^that X*^^ accounts for 25 percent of the variance in 
^the^SQOres of P^, and accounts for iSjpercent of the variance In the .scores. 
Similarly j if wfe read in 'the seeorid row from P ve note: that X (modeling), ' 



all correlated 



^ (positive reinforcement), and X^ (negative reinlforceihent) • are £ 

significantly with . Again^we can squar^* each of. thesje correlation coef f ipients 

unted fot by each of these variables: 
variance as j does X, (negatiye / 



to determine the^percentage of li^rianc^- acc 
X- X^o4eHng)» accounts for 12 percent of thfe 

: / " • 

retnforcenj^t-)'; but X (positive ^reinf or cem€|y:) ^fcpounts 

)' 

yariancd. 

• We' hSve two predictors of B^jr^X^^ 



and X 



; and thr^e p 



for 20;fpercent of the 



redicilfors of V2"y^y 



X^ and X^v But note that P^^ and-V2 are^;^ubsi:antially correlat^ (r ^ .78) . 

^and that X- is, correlated, wit*h X^, X^ and ^X, even though i:he correlations are . • 

^Tiot statistically significant. X^ has a significant correlation with X2. 

^ r ' . 

In general most of the^ t^achet performance variables are intercjorrelated well 

*' • ' ' ' / ' ■ J ^ ^ \ 

above a zero correlation. These relations suggest certain interdependencles 

ainong the performances which- are revealed in the correlation matrix. ' 

* ' ' • ' ^ ' * - ' ^* 

The methods of .multiple regression analysis. use the data provided in 

a matrix such as that presented above. The calculations utilize all the 

, , ■ ' ' . , ♦ - - ^ ' 

correlations ^mong the variables. ^ The tedhnical analysis produces two, basic 

^ ^ " • ^ '^'^ • ! ' ^ ^ - - ' ' • * 

places of Information, (1) a midtiple R and (2) beta "weights. The mtiltlpl^ U j^^l 



a correlation coefficient and is interpretfe4 ii? the *saihe w: 
of correlation coefficients th^t we have previously /discusteed; that is, 

2 y-^'^' ' f,.' • • ,•, , : ,r .„ . ; ■ 

*'R , ttanslate^ 4nto a/ percentage, re'^resents the per,centage pf var-iance 
' : predicted by all tha p'redictor variables entered into thef equation. From 
^this^figur^ we knoW how, useful the array of preidicto^ variables is in 
' accotlntint for th^ variance in the /dependent variable, in this case a 
proficieKj/ scord^ The ' beta weigt/ts are Standard partd/al regression cd- * 

efficients,^' which are ratios va(ribus ^combinations of /the original or iero- 

' ■ 1 ' ' ' ' / ■ /^'-/' ' / ' ■ ' ' ' - . 

order correlation coefficients. / These beta ;w6ight8 are' then transformed . 

into- the coef f icienjrs -in the, prediction equation by; multiplying the ,beta } 



■'^ „ , 

♦ weights by the r^itio of i^he variances of thfe variaBle^ involved. What 
each bf-^these coefficients represents is tW weight ^cfcached«.to a particular 
variable Independent of the ;otner, variables ; «hey:af^ partial regression ^,; , 
caefficieritsZ-rf These weights var^^ln magnitude ^jid may be ^either positive ^ 



-7 '• 



ox negative ^Ln sign. A coefficient which is negatixe. in sign means that 

i' ■ A- ■ '■ ■ ■-' ■■\. " f • . ■ • • '■„■■ ■■■•■( 'i 

^variable ysubtracts JErom the prjedictiani n 



Consider nw the following e£ti*ation: 



I- 



u . 




(I) ?^ 



^1' 1^2 ^2 -^^^3 




In. this equation P2 '^'^ th$. ii^f t^and side of the equation is* the set of ^c;ores- 
^ £[Vptof^€^ency which are .to l^e^eclicte4 l>y the^vai^bles dn'the^i^^-harid / 
side of the equation. Notice that. P^^ .is included InTthe predifction equation.-: 
13ii ;r)elSSoV^6r Incl-udlhg P- is that- it is highlV correlate/* witih P^ ahd is* 
' -^ therefore, a predictor of The X's represent th^rfour teaching performance. 

; variabi^^hat we.haVe been talkihg^about, and th6 coefficients of these 
values are. represented by- the letter , b» These coeffilcients ^e the ^regression 
weights that we discussed above. Each of these would have a numerical value-. 
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calculated * from th^ correration matri::'; 
up and/ we have not actually cal.cul^^ted 



4^1 



and. may I have a negative sign. (ActtialtValues of these cqef f icieatis^oulti^ be 



the numbers in t;he matrix were m^^de 

weights from that matrix.") 

\ ^or .this equation there will k| multiple R.- Assume tji'at for this 

equation the multiple R is .6A;I applying the laethod described above, we- 

square/. 64; R = .41; .41 x 100% = 41 percent^ The empirical conclusio^ /to 

be, drawn from these ^ata is thai: the four predictor variab^es^ (tKe foi 

.teaching performance variables) and the other proficient variable prf^lct 

41 percent of the varianpe^ in the scores. 5ince 100 percent "of ttf^ variance 

Is to be accounted for, the^e. five variables account for less thai^ Half ojf t;He 

^variation in the sccTres. .1 • ' 

^^.^ You may a$k what else Is related tb variation in the proficicfncy measure? 

pur equation only includes four kinds of teaching performances^ arid et^ measure 

oi^iAitial proficiency. The remairrin?g variance may^ be predift^by such 
♦ r I ' ' ! • ■ . . T • ^ - .1^1 ' 

,f aict^<>rs as the previous experience) of the students with English, how long ^ 

th^y^\have been in 'the United States, 'the kinds, of' communities /in which they 

I , . * - o - / / ^ 



are\cWrently living, * their educational aJttainments, and' theiyr past and present 

\ A . t ' ' ' / I I - • * 

^the prediction 

/' . ^ 

equation^ first havlhg found the . cori;ei ation of suchVariaW.es with the / 
performnce variables and the p^roficienjcy measures,' we wotfLd find out if the . 
additior^\f\ these* variables ac(^^^ for sbme or all It tte remaining variance. . 



If V»ki.tef natively, w^ assume that R for equation/ (Ifl is .87; 'then 



R = -.76; \6 \ 100% = 76 percent.* In this case the ' teaching performance 



:^ variables andXth^i .prof iciencs?^^^^ /76, percent o.f the • 



variance in theNP^ N^cores. This R tells us that t 



he predictor variables 



are very useful itk predicting* vMr^^^^ in the.|rcjflciency scores *(P2) 
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r / 

Again, by adding Other p^redictor variables more .of the variance may be 
accounted for. In this example there is only*one-foiirth of the remaining 
variance to be accounted for, and some of this variance will be accounted - 
for by unreliability in the measures of the predictor variables, (We will . ' 
return to' this, point below. ) ' * . 

2 * ' 

The R .value tells the researcher how usef his, or Tier ^rray of predictor 
variables are for predicting the differences among students on the achievement 
measui^e. .As the reader can perceive, unless the researcher has knowledge frojn 



previous research or from theory, h^e pr she could select a set of predictor 
variables whicli have little utility for predicting tt^e variation on the outcome 
measure.. In developing. a research study in which multipfe regression analyses 
^methods are to be u^d, a rational way to pfxoceed l^^to decide on those factors, 
^most ^ikely to be related, to the outcome measure. Even when previous knowledge 
suggejBfcs that a' relatioi^hip between a predictor variable and an outcome 

measure is likely to.be found^- it may tym out that' in a particular . sample of 

♦ . . ' ' ' * . ' / ' ^ - ~ 

observed data that the predictor variable does not relate to the ouj^come measure 

in Jthe same way that^it has in previous , studies. / . - ^ 

The logical method underlying the use of multiple regression analysis * 
' - , . /< / . ' ' ^ 

in research is J.nductive in character. The researcher enters as many variables * 

■ . • ' • > ^. 

into a prediction 'equation £^is feasible in terms of the practical matter of 
gathering data on those variables. The' regression equation' tells the researcher 
Kow good the *particuiar: combination of variables that he qr she I>as selected 
is In predicting the variance in the outcome measure. The aighs of the 
coefficients of each of thefee variables tells him or her how the variable j 
contributes to the prediction either ad4itively or subtwractively. ^ The numerical 



size of the coefficient tells the researcher how inuch x^'eight each variable 

has in the prediction; also the statistical significance of the regressidn 

weights tells, him or her hgw reliable a predictor that^^arlable is. 

Let us illustrate these ideas with our made-up example'; assume that * 
• ; • / ^ ' ^ * • ' ; ' ' * ' 

the j:alculat ions yield the following equation (which /i3 the preceding equation 

^» • - • ^ * ■ ' , • ' ** ^, - 

with numbers. substituted for thfe b^s). 

(2) P^^^ 1.20 + .98 +; 1.05 + 1.25 - .'75 X^ ' 

^ ' ■ * 

In this prediction "equation note the following: (1) the largest numerical 

' ' ,^ ' . / • : ' — ' ' - 

weight is associated with P , the next largest with X (positive reinforcement), 

^ ' » \ %. 

the third larg^ weight, associated with X2 "(negative reinforcement). Assume^ 

that all of thqjsfe'coafficieats are statistically significant.' 

^ ' , ^ ' , * / ^''^ • 

What this equation tells us is^ that t;he Uest- predictor of the me'asure of 

oral proficiency, P^, is the other measure of^oral proficiency (a, result , 

i ' ^ , ~ \ • • 

vhich is not surprising). Among the* four performance varil^ables we notice that 

negative reinforcement ■ (X^) contributes negatively to the -^ediction; that is,, 

if we were 'Using only this predictor variable, we would| predict lower scores 

for students that had teachers who ased more negative r^i^iforcemement. in 

contrast, X^ (positive retriforcerantj has a larger and Jjositlve .weight, it 

we considered bnly this variable, we would predict higher scores for students 

who jiad .teachers that used more positive reinfoircement. 

>.Bu,t,^ ^^^^ourse, the purpose of using m^iltlplfe regression methads is 

to consider all of the variables together and not confine .tie analysis to the 

predictive power of each variable by*.itself. -l^hat the prediction equation 

represents is a set .of data from which one may theorlzfe about the celative 

influence of a, variety of factors. The logic of this Inductive analysis 



proceeds as follows. .First one looks, at the sijte and significance of R^.*^ 

If R is significant, the first requirement, then its magnitude is next 

considered • If^ the magnitude is not large, that is, a relatively .smaller. , 
* . ^ * • ^ ^ • . 

proportion of variance has been accounted for, know that our predictor 
variables do not account for most of the variance in the outcome measure. 
One reasonable conclusion is that there may be other predictor variables 
which when entered into* the equation, improve the prediction. Even if 
is^ smair, one may theorize about the variables that are in the equation; 
but, obviously one .is likely ^tt> jlevelop- an ins^omplete theory becausfe the 
varjAbles in 'the' theory only account for a small proportion of ,the variability 

, in the phenomena to be predicted* . ' * . 

If , in contrast, -the multiple R is relatively large, then onehas.moVe 
confidence that the v^^iables in .the prediction equation represent a, dyiparativel 
splid basis for theorizing 'about the nature of tfife phenomena involved in the 

^re^iictioft. Suppose, as we did above, th^t R is large so that th^tt*ediction 

is accounting for" about 75 percent of the. variance. It is notf unreasonable to 

\ ^ * * ' 

, infer that we have a^et of var4.ablea which, constitute a jeeasoriabXe basis for 

• • , • ' , /^"^r ^ . 

developing a theory of instruction. ^ ' 

Referring, back to -the equation that he hav^ been • discussing, our theorizifig.. 

might go something like this. Since P^.l)6s^the largest positive regression ' 

•weight, we may infer that, the initial proficiency status off'the student is, a^* * 

relatively critical factor in determining what the students* finals achieyement 

is likely to be. This idea makes ser^e" since new learning is always built . ^ 

upon or int?ergrated into or reorganized witlj previous learning.. In this 

instance we are studying the acquisition of language; we are r^son^^ly * 



■ ■ r - . , 

eerdain ott theoretical" grounds, .that the level of ^acquisition previously 

attained is. a base upon whifh subsequent acquisitidn mi^t be developed. 

The empirical data, as represented in the prediction equation,- corresponds • 
^ ' . . . •* . '* ' ' 

to our theoretical expectatio];^. ; - ^ . . ^ 

«* ' • • ' 

Next, we noticed .that X^^dnd X both, have large (in the^'cqntext of 

: - ' . ' ^ 

this equation) ^positive regression weights. ^ These two variables are ttie ^ J , 

' . ' * • " ^ ■ I- — - 

practice — :positive reinforc^mgnt variables .^^ .We would speculate or - hypothesize 
that teachers who^provide ittbife , practice in. using English and positive rein-^ , - 

S , ^ > " . > . ^ ■ ' 

forcement for appropriate language responses Have created an instructional 

model which is likely to be associated with higher achievement scores. But ^ 

we'^also notice*d that Xw" the modeling, varxabje, has a. positive regressioi;i 

weight near in fnagnitud.e- to that of .X^. .We»in£er that .4he provi§ion of modeling 

i^-a fax:^tor which al^o si^ificantly a ffec ts predicted scores. • < ' . 

At this poi|it tt^ the inductive analysis we are t;eason2^1^;si>ie which 

variables are likely to be associated with'^hlgher scores and w^iich- are not. 

NoW' we should consider the limitations of t^ie method to prevent ourselves 

from drawing inferences tbdt ^re-too' strong and Vhat go .,beyond the P9wer 

of the me|:hod. We are making infer^ences fi"om''one study. The geheralizability 

and stability of the results are unknown^ But, sinc6 in this ^cai^e-out 
* • ; . . • • 

inferences also, make theoretical sense, the*risic in applying the results ^does' 

, • . ^ ' 'r ^ ■ 

not seem great.- , • ^ v - ' ' 

We might also -be tempted, iht example, to sdy 'that practice is the most 
import;ant variable (after' that "of tTie students' initial status) because it . 
has the largest regression -weight. Bift we mqst remember that-.ve have only 



this one prediction "equation. -It' has hot Jbeen cross-validated in other 
^studies, and we know that these regression weights like other statistical* 
phenomena vary in their magnitudes in 'different samples. So we caftnot 

coiteliide that positive reiTiforcement is the most critical variable and that * 

' * ' X - ' ^ • ^ 

modeling is relatively less important. 'All vze can say is that we hkve a 

basis for hypothesizing that three of ftie'petfornjande v^riabLaa^are positive 
. ' « ' ' - ' • -\ ' ^ 

-^^^ ' - • 

• predictors and that a fourth variable is a. negative ^•predictor. Note, however, ' 

chat ftiese variables are, also i the critical vaAables^n social learning theory 

an d reinfo r cement theorie s. Therefore,^ we can have r^sonable confidence *(from 

theory and^our data) that we have id entif ied variable's which may be critical 

In the acquisition of proficiency in spea^king English* 

W6 Jiave spent several pages going-through the concepts af the methods 

. of multiple regression because they are the methods which we have used in l. 

the researcli reported here. While we have stressed in the more technical 

chapters the limitations and qualifications of the r^esults of the* analyses, 

we, have taken pains here to illustrate how, within those empirical limitations, 

^ one jcan begin to reason to a set of ideas , or h3rpotheses or even ^ theory 



about. the, nature of the instruction that is *likely to be effectiye, 

♦ Applying the Results of the .Research , . ' 

» " ' ** , ^ 

The obvious questictn for the prac.titioAer who wishes to use the results 

of an^empirical study of this kind is "How do I* apply these results to design^ 

instructional strategies?" Should the practitioner, fpr example, recommend 

.to teachers of English as a second i^giiage that they use more positive 

relhf or cement in their ;teaching, or that they increase the amount of modeling 



they do for studenis?; 6nder^ying these questions Is an epistefological 



question. May we \v,e causal inferences from correlational 'data ♦of the 

kind produced in a ttultivariate analysds of teaching perfolfmance? * — 

A Correlation between two-. variables does not imply th^t, one variable 

is necessarily the cause df the other. All that we learn from the correlation ^ 

is that the two variables are associated. When we use paftial, coi;relatibnal ^ 

methods,* however i we afe^ separating out the relaj;rlye ^influences of several :/ - \ 

variables* Iflien Ve partial out initial proficiency scores to find the 

correlation between^ teaching^ performance and final proficiency scores, we 

* *' ■ \ 

estimate the relative influence of initial proficiency and .teaching perfcr&ance* 

^ - - * ; 

We still cannot make- a strict causal inference that the teaching performance 

variable "causes" ttte change from initial status to final status^ but by a 
process of reasoning we can estimate the plausibility of this possible causal 
relation* WQ^.know t^at the proficiency of a stddent .at the time instruction 

* begins "was^cqiJired before the actual instruction. • -Therefbre, when we find \ " 
a teaching performance tVat i^ correlated with final status sco;^^^'af t^r ^ 
partialling out initial scores), we may infer that the t_eaching performance 

. has adiedTto proficiency over and above what the ^^P^ent initially has acquired^ 
This inference, *however, is only one of several .£hat ^ky bjff^made. It . 
may be that characteristics of the students help them to.,^e more responsive 
to instruction "and this responsiveness to the teaching per cormahces accounts 
'for the gain in proficiency/ Or, ther^ may be a relation between thfe' level 
of performance acquired prior to instruction ^and the effectiveness of particular 
methods of instruction.- ^ : - . ^ • 
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Tjie methods Wed in' thi/^ study* should help sort out the likelihood ^" 
that otlfe these, inferences i.s more valid thdn .another. ' But strictly 
sjTeaking no conclusion aUout ' the relative validity of these different 
hypotheses" can be drawn from the* data of the study itlself. Only by a- process 
of '^analysis and reasoning can one estimate a likelihood of one explanation 
being more valid than another, but the conclusion about, such an interpretation 

Us a. hypothesis which needs to be tested. ^ ' i ' 

' • f ^ ? • ' , ^ ' . . ' \. ' 

The only way we. can really know Which predictor variables directly; produCie 

a\change In profic'iency \s by creating an^experiment in' which, these predictor, 

vati*ables are m^ipulated in a systematic way/ The next- step, after a study 

! of this kind. Would, be' to design^ an'experim^nt . in which teachers are trained 

to Vise those teaching performances which* are significantly correlated with \ 

fitfti proficiency, and in which the effect of these- increases, in the teaching 
. 1 ^ ' ' * 

perflormatices on proficiency can be measured. Only if we conduct a study in 

Ijihlcn w§ sj^stematicali^y control 'some of •the variables so that we cain estimate 

the tffects other vatiables^ can we draw a strict conclusion about which 

J[4ri4bles have a causal* effect upon proficiendy and which do not, and the 

r^latjlye influence of eacli potential causal variable. \ ' . 

le practitioner^^may not' always be in a position 'to conduct such an 

aent or to. have it conducted for* him or^ her,, and therefore wants to 



that 



Is 



w tjiey should proceed -when they have reteivpd data of the kind produced 
3-study# One way to proceed is to make .inferences atout pot^tial ^ 
relations and to act on these inferences. If we have found, for example*^ 
particular pattern of ^teaching performances lis. cor|:elated witll outcome , 
Ls we can urge, teachers to use these methods or train them to use them, 
(in gather -data systematically to see how^uch * improvement in proficiency 
iially achieved. • 217 *' • * ' ' 



r 



. At the end! of* this chapter, after we*hav6 pr^seflte'd the major results 

* ' ^ • "^ ^r- ' 

of this .study (which' are more comlkex, thau the exaftfplea l^p 'have been using) 

' V - . • ' / ' * . ♦ 

we will make s6me:Ssuggestlons» about practical -action. -For the more technically- 

\ ^\ ' ,\ . V ^ ^ ^V- . ' " • . ^ ' ' ' ^ 

minde^ reader/ we^ state th^t we are obviously reasoning beyond the ^data to 

make iriferen9es about potential causality which might lead to programs of action. 

Ebr the practitioner we are suggesting what m^.ght b*e tried and ways^ es^ii^iating 
/ ' . . . ^ , ' . ^ • * ^ _ . ' ' ^ 

'Whether, vhen* these actions are. taken,^ they in fact have a significant effect..' 

• * The Selection of Predictor Vai?iables, ' * , ^ * ' 



^ In T)6rlorming a study of the kin^ 'described- here the first step is 1fO' 
select variables, likely to \be .predictors of , proficiency in, speaking -English* 
Since there i^ ,no formal theory that can guide lis in ^he selection of ^the^e 

, variables,- but there is some relevant information and some, generally accep.ted 

• " /• ' • ' ^ 

ideas, the .investigator |>roceeds by collecting as much information a& is 

feasible about a range of predictor varia]3les, We^a^e reasonably certain,, for' 

• ' . . ' ' ' . * ' • . ' ^ * 

example, that the initial prtoficiency of the student is likely to be^eJLa fed 

to final status because this ^result has Jjeen .found in a very^ large number of - 

V f ' ' ' ^ ' ^ ^ ' ^ ' -^Z ^ y J: 

studies On instruction. This select^-on also makes theoretical sense because 

the acquisition oif language. Is p^rt the acquisition of discrete responses 

which are progressiveJLy integrated into more coplplex patterns of responses; 

that is, th^ learner acquires new responses .to some .degree.-^y .using previously 

learned responses. ^ * ^ %^ ' ' 

Wer^^ould also expect cbaracteris|:iqs of .the students to be related/ to ' 

how.wel|L tJhey learned. Unfortunately in studies of this kind it is not . 

usually possible to gather as wid^ a railge/ of * infor^tion on students'^ 



chafacliejastjcs -as^ might desirable, Frequ^tlyT^ in this~"case, we 
have to settle fot desdriptors of characteristkcs which are proxies for ^ 
underlying characteristics* A student^ s motivaticfh to acquire new language, . 
for example,* is likely^ to be an important Variable, but '-it ±b not easy to ' 
measure this motivation ^rectly^. Therefor^, we resort to gathering 
information 'about, the student which are indirect measures of motivation 
such as the amoimt of education they have attained, or whether or, not they 
studied English in their country of origin. These measures are obviously 
indirect measures but a reasonable case can be made that a student who* proceeds 
further in the educational system probably has morfe motivation to be* educated 
or' has attitudes towards education that stimulate them, to seek more learning. 
The reader Jt^ill recognize the .relative weakness of a proxy of this kin^y 
thinking that a student who has not progressed very far in the educational 



system may hot have had the economic resources to do so. 'But we are using the 
indicator to* find out if a variable that may be an indicator of motivation 

is related to student performai^ce. If it is,, furtlliet exploration can be made 

I ' ' ' ^ ' 

to* pinpoint the precise nature of the underlying cfiWaCteristic whiCh may be 
affecting the student's responsiveness to instruction or which helps him or 
heV to become ''more proficient. - ^ * * > 

In this study we have used t^ree ma^or categories o^ predictor variables. 
Th6 first of these is the initial performance of students, that is,, h6w they 
scored on different measures of proficiency in English at the -beginning' of 
the research period. The students ^nter Che Center with varying degrees of 
%bility to speak English, fhe^r^ fact, are placed in language, classes on 
the .bai^id Of their measured periortiance in understanding and usage of English. 
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, ^ . _ . „ * ^ . • o 

We* have, .therefor^, included, among' the* priedictor variables tKose measures 

* » < ^ • • • 

or prof iciency that were used at the beginning of the research period or" 

• * • * ' ' * 

that were used when the students wete admitted to and placed in*, classes in 
the Center • \ - * . • * 

Another class of variables are those, whrch describe, the previous 
experience of the Vtudent, Since t;hi& study uses adults as, its sample of 

students Ve know that we are studying individuals who have a relatively long 

v * ' - 

learning history. It is reksonable to expect that various kinds of influences 

on that learning history would be related to how the students acquire greater . 

proficienc>r while in the Adult Learning Centei?. I 

We collected data on a, variety of background fd^ors such as the amount 

\ • • ' 

of education that the students had, 'the amount of direct experience wi*th 

" • , t . " * 

English, both here and in their country of origin, their age, the kind of 



/eric; 



employment they held in their country of origin an^d that they ma^^be holding 
here. ' We would expect^ variables like these to be indicators of previous 
learnihg^history.^ .Amount of education, for example, is a prdxy for such • 

variables as^bill.ty- and motivation. We know that students who proceed 

\, 

further in the edticatioiM. system have to have achieved successfully. as 

^.they progress through tills s^tem. We also assume that those who have 

proceeded. further in the system, particularly when the age for compulsory 

education is lower in the country, may have more motivation* to become 

educated (allowing fot^..dli f ererfces in economic resources to allocate to 

one* 8, •education). Note that we are, not claituing - that such variables are 

ft ' • « ' - 

causes Vof proficiency in speaking English or^causeg of educational motivation 

ox attainment. We are simply saying that* these are indicators or proxies 

worth entering into'^a multiple ragres.sion analysis to see which predict 

-acqliired proficiency In English. ^ - 



The- third category of predictor variables are tbose related" to how the 

' ' m ^ ' ' . ' 

teachers actually teach. ^ We obtained .these data by directly observing the , 

clasfses in the A'd'ult Learning Center* Our assumption was that different 

methods may be more or less effective in facilitating the acquisition \)f 

proficiency in speaking' English. ^ ' ^ 

- ^ J ' . • . , 

In this study, of course, wfe may analyse only those teaching performances 
which actually were observed. It is conceivable that t^iere dre other waj^rs 
of teaching English'as a^'second language "that TSi'gfTt be*rmore effective than 
the methods that we have observed. Thus, one bf^ the major limitations of 
this kind of study is that we can find out only if the observed performances 
are more or less effective. ' , . 

In selecting the^ major categories of predictor variables we have picked 
those most likely to be predictors of subsequent ptoficiency. There may be 



other variables that^are more powerful that we do not presently suspect to be 
powerful prediator^^. We have chosen .those which on the* basis of ^previous 
research and theoretical knowledge are most likely to relate to learning of 
this ki^d. 

^ " ' ^ Major Questions^-^of This -Study' ' • ' ^ 

The major question of this study is,, "How mucli difference do the methods 
of teaching make in influencing the acquisition of prcff iciency in speaking 
English?*' As we h^ve pointed out, we cannot give a strictly causal answer 
to that question; tfte question we do answer is, "How well do teaching - per formanc.e 
variables predict proficiency when. We allow for the influenc'e of other variables 
on proficiency?*-* - / ^ • . ' ' ' * 



. We are .also intferested in icientifying those teaching, performances 
which aret most effective* Teaching performance are complexes of a variety 
of behaviors ot actions which are organized in many different ways by each" 
teacher. In th^^^ult Learning Center we bbserved teachers who .frequently 
modeled the desired skeaking perf orman<:ev^for the students; 'we observed 
teachers who asked questions that stimulated the students to practice 
structures of 't?he English language; we observed varying amounts of corrective 
feedbacl^; we observed teachers who stimulated students to sp-eak English 
discursively and did very little modelling of the appropriate responses. Ar^ 
sofme of these ways of teaching mpre effective than others?' , ^ 

A^third question is related to the consistency o'f, the procedures used 

* / f ' ^ 

by the teachers. No teacher .teaches in. exactly the same way on every occasion. 
But *there are .two aspects to this variability of the teachers* performances. 
A teacher may have an overall sTyle 'wh ion [varies somewhat from occasion-to- 
occasion. Or, a teacher may have combinations of different kinds of styles 

* * ' * J- * ** 

which he or she organizes differently from day-to-da^y.. In the field of 
teaching English as a ^ecfond language there are at least two major methodological 
styles, the audit>lingual method and the ''silent w.ay." These two styles- in 
their ideal form differ radically. We know that different teachers^ In the' 
Center prefer one style to another, but there are also teachers at the Center ^ 
who use mixtures of these styles^. * . . , 

' A major problem in this study was to identify the characteristic styles 
of'each teacher, and to^^^^dy howt consistent they wer^within a style'or how 
they combine^scyles and how they varied t;heir teaching from occasion-to-occasion. 

ft is possible that there is so much variability in a teacher's style that it 

. • . ' " ^ - . / ' ' 

would not be possible to ideptif/^ny one teaching performance oi^ even a 

^ collection of teaching p.erforman'ces which had a systematic effect on proficiency. 



(THe data presented in Chapter III are the resul't^ of our analyses of the 
teaching styles, used in the Center,. That chapter also contains information , . 
on the consistency or^ yariability in the styles' use<j by the teachers.)* /" 

Although th6 major goal of this study was to study the teaching performances 
or methods of instruction, it was also necessary t;o askyquestions about the . 

. " ' • , r • 

influence of other variables on acquired proficiency. For this reason, as 
we noted abqve, we have included in our analyses other -potential. predictor 
variables related to the characteristics of the students, both their background 
characteristics and their initial status as measured-at the beginning of the 
research period. Two maj9r questions are,' therefore: "How well do descriptors 
'of student's background' characteristics predict ac|quired proticien^s^^ and 
"How well does the student's initial proficiency predict his or her final 
proficiency?" . 

As can be seen, the major questions of the study ^ask which of the various 
categories of variables predict best the final proficiency of the students/ 
It will *be useful to consider at this point the potential implications of 
various kinds of answers so that the reader can begin to anticipate how to • 
use the. data as he or she progresses througl^ ^he summaries of it in this 
chapter. . v . 



Possibilities for Applying -the Research 



Cotisider how the results of- this study may have come out arid what the 
implications" of these results might be. Suppose that teaching performances 
x/ere foun* tq be nonsignificant predictors of acquired proficiency. Would, 
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this sT^suirt idiply t;hat the methods used in the Centjer 'were 'ineffective? 
Not , necessarily. It may be that the methods, while; tl^ey "added nothing" ' ' 
to thfe prediction of proficiency may in fact be thel methods required for the 
characteristics of the studeniis to have their effect op. acquiring proficiency. 

cannot automatically assume that because the ^nethods)^ bein§ used are not 
good *predic tors of final proficiency tKat ^they were not useful in some other 



way. 



Suppose that we find that no single style of teaching is. particularly 



effective,' that is, none of them is*a good predictor of f inal^acquired 

proficiency. This result seems to have more practical meaning because it 

suggests that strict adherence to any one method is not in and' of itself 

likely to be sufficient to produce a substantial effecft. UnfortOriately , the 

method t)f the study does not permit us to draw a strict conclusion about the,, 

effectiveness pf one method over another for several reasons: (1) there are 

very few, instances of a method belng*used in its.»ideal form; (2) there are 

too few instances of teachers adheri^hg to a method make a strict comparison. 

At best we can say ^f rom this study that adhering more ''or less strictly \to a 

method did or did not^ produce greater ^proficiency in speaking English, but 

this result may ^elX be contaminated by characteristics of the teacher which 
* ' • . ^ y / ^ * ' * 

interacted with the method. • ' ' , V^, 

* The staff of the Center may examine the purer forms of a method being 
, used by particular teachers to see if the use of the method could be Improved.. 
,0r the staff can study other characteristics. of the t^dchers which may be 
interacting with the use of the method. . ^. * 
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' Anoi^r tuajot possibility -is that the characteristfcs of the students 
are the best predictors of their acquired proficiency. This result leads 
to,sev^ei?al different kinds of possibilities. One poss.ible course of action 
is to find out .if - teaching methods interact with characteristics, that is,'' 
are' there some teaching performances or methods 'whicJh in combination with 
characteristics of. the students are associated with greater achieved proficiency? 
If such were the case, one could be more presjcjr^-ptive about the organization 
of classes 'and the appropriate methods to be used in .the different classes • 

These are different*^ ways in which the, results of the stydy may com^. 

<\ ^ " ^ '• 

out, and what the implications might be;^for action if Such results were obtained. 

The reader should be aware that in this type^ of study we are trying to estimate ^* 

wM:ch variables predict best acquired proficiency in speaking English. ?r6m 

the data produced by the study,- it will not be possible to draw strict causal 

Inferences that these variables in fact made the critical ""diffeiience in the - . 

acquisition of prof itfiency.. But the .results will suggest some possibilities 

for action. These possibilities should be regarded as hypotheses about pr 

theories of instruction that might be tested subsequently. , - 

We now tum^to summarizing' the details of the\study itsej-f, and at the 

^ end of the chapter we wilL.ag^lfis engage in an analjrsis of the possible mdaning^ 

of^these results and, the possibilities for practical action. ' V ' 
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THE SAMPLE OP, CLASSES STUDIED ' " 

■ • • ••• . - 

The Adult Learning Center in^West New York provides train:^ In English . 
for ^students- whose native language is some other language* The students are 
adults^ ^s the name of^ the- Cent.fer indicates, ranging in age from 3.9. to 73. 
The majority of the stu3ents "^Ste* immigrants from Cuba; most of tfie other ^ 
Students are from Caribbe^ and South American countries, *o that t;he native 
language of over/ 90 percent of the students is Spanish.* 

The classdia are conducted both during the day and at night. The day-school 

• \ ' / s 

classes are held at the Adult Reaming Center which is located, in an office 
building in down town West New York. The night-school classes are .conducted at 
Memorial lligh School in West New Tojrk. The class scljf^dule follows the pii&lic 
school calendar. ' ' . . ^ * . 

Th^e majority of classes in the .4^v,^ jfih ooymet for aiv hour-and-ra-half a 
day, five days a wedk. * Of night-school classes studied flv^ met for a _ 
two-hour" session thr^e evenings a week; the 'sixth class met only two evenings 
a. week. 

All of. the classes in the day school were studied, with-. one exceptions 
which was dropped from the study because the teacher transferred to another, 
position. T^ere were 14, day-school classes taiight by_six different teachers. 

^ There were 22 classes available for study in the. night school of which 

■ - - ■ . 
^>six different classes. with six different teachers Were selected by applying' 

the following criteria:^ (1) the teachers did not also teach in the day school; 

(2) .their clcissrooms had a sufficient , number of "r^epresentative "students"; 

(3) there was a range, among classea in the level -of student proficiency in 
speaking E^lish; and (4) tliere was a range ajpong teachers in pr^vious'^traiikng 



and axpeT:ienG>e. The representativeness of the classes was "k'ssessed by, 

I ' '** " . ' • ' • 

> 

gathering data on the students, then describing 'each class in terjns of these* 
characteristics, and then ""binding a set of ^classes where the class had the 
modal characteristics of the students in, the entir.e sec of 22 classjes; 

StudentkS provided backgfpund information on 10 characteristics^ their 
sejc*/ whether or not they had received a high school diploma, their age, the 
jf&ducational level attained, 'the , amount of *time tb^y had be^n in the United 
Scutes, the amount of English they had. studied in their former country, the 
amount of English they had studied in the United States, Jiheir country of 
origin, their occupation in their former country, and their occupation in 
the, tftiiteJ States.. Ta1)le^2,l and Table 2.2 provide the summary statistics 
on these characteristics of the students. (These tables a:^a reproduc.^d from 
Chapter 2, and the reader may* refer to that .chapter for a more detaile,d 
discussion of the students' back^^^und characteristics, ) , * ^ 

' "The background informatit>«, was collected in the day scho<5i^on 1^8 students 
^ the beginhing of the study; however, a .number" of these students dropped out 
of the Center before thq completion of the study (t4iere were 81 of t^^original 
.students at the completion of the study; tVe, attrition rate was 45.^^^cent ^ 
between November and June for day-school studlinfts). Table 2.1', therefore, 

presents two sets of data. The data in Tabl^ 2.1 under the heading " Matched" 

* ' . ' ' ^ ' 

is the data on students who were present during -the entire period qf the. study, 

* / * • * 

* - o <• * 

that is, from November"* to^ June* A comparison was made between the original 

samplft^bf students present .*in tMb Center in N^^Vember, and the sample remaining 

in June, tlo significant differences between the two samples were found. 

• ■ ., ■ . '\ ^ \ ■ 
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TAn'.i: 2.1 



Day Scliaol Student Ra« A ground FTlformation; — 
Pretext and Matched Samples: l^escrlptive Se?itistics 



Number 

Sex: M 
F 

Diploma: 



Yes 
No 



^.Age: Mean 
Range 

. Education! 

Mean Years 
Range 

Time in IS. S* 
Mean ^Years 
Range 

Former English 
^Mean Years 
Ra^g? 

English in S. 
.* Mean Years 
Range - 

Former Country 

' " Columbia 
Cuba 

Dominican R* 

Ecuador 

Peru 



t 




0 

Pretest' 








Matched 








Total 
Frtfq. P<?rcent 


. Fr?q. by Levjl ^ 

"12 3 


Freq. Perc<«at> 


Frcq.» oy I.c^fel 

1 - 2.. ; 3/ 




U8 


■ \ 


52 . 


42 


5A 


81 ^ 




24 . 


28 

•r 






5A 


36 
63 


^22 
' 30 


21 
21 


-11 
^A3 


. 26 


32 " 
68 ' 


9 
15 


<^3 
15 


4 * 
25 




• 63 

83 


A3 
56 


17 
3A 


15 


,31 


32/ 
A9 . 


. Aot * 

;60. ^ 


8 

. 16 ' 


10 
18 


14 ■ 
15 




A5 
19-73 




A6* 
21-69 

m 


. AA ' 
19-70 


AA 
2A-73 


' 1 
A6 
19-70 




48 
21-69 


45' 
19-70 


2*4-67 




10 
3-17 




9 

3-16 


10 . 
A-16 


12 • 
A-17 


10 
A-16 




- 9 
4-16 


XO 
4-f6' 


11 - 
4-16 




6 




5 


. 6 


' t 7 ^ 


6 


i , 


6 


' 6 7 ' 





1-24 



1-15 • 1-22 



^ 1/08 ^ .58 ^ .83. 1.83 
0-12 0-5 0-6 5hl2 



1.08 I 

'0-12 



• 50 1.41 1.33 
0-J.'.5 0-12.0 .0-3^ 



Other 




1-24 



1-15 









1,17 ' 




.83 


0-12 




0-5 


'1 
. 1.17 




.58 


0-3^0' 




0-1.5 


' / 

"3 - 


4 


1 

1 


73 


90 


22 


1 


- 1 


0 


1^1 






0 . 






4 






Matche;3 , 




1 


2 


0/0. 


0/a 


0/0 


-1/0 'I 


1/^ 


0/0 


' 6/20 


4/4.> 


2/7 


°' 0/0- 


0/0' 


0/0- 


0/0 


,0/0 


b/0 


' 2/7 


0/2. 


2/5 


O'/l 


•d/0 


0/1 


• 23/1 


5/0^ 


8/0 


.4/1 


0/0 


. 1/0 


0/0 


0/0 ' 


\0/S) 


' 1/0. 


1/0 


0/6 


1/0 


m 


e/o 


1/1 


0/0 • 


i/o 


19/7 


r/o 


• 6/4 


. 4/6 


HO 


•1/3^ 


13/17 


'373 


.4/2 


•0/19 


0/12 


> 0/6 


6/i 


1/1. 


. 3/6 









1-17 ' 1-24 
\ 

,75/ ^1.83 
(f-2*^i-^ 0-12 



1.25 



2 
25 
.0 
0 
0 



1.50 



0. 
26 
1 
0 

0- 



Occupation 

1. Foreman 

2. Craftsman . 

3. Semi-skilled Worker 

4. Jl^bprer 

5. Household Worker 

6. Personal Service 
7* Flreman/Policcraan 

8. Professional 

9« * Technician 

10. Farmer ' 

11. Farm Worker 

12. Business Owner ^ 

13. 'Managcr/0,f flcial 

14. Office Worker 

15. Salesperson ' \ 
' 16. Housewife . . 
"^17. Unemployed' 

18. ^tudeht 



, Previous country/USA ^ * r\ ^\ 

See Appendix A fo.r definitions of categories.. ^ C*^ "2 



0/0 
• 0/0 

0J9^ 
0/0 
0/0 
0/0 

0/0 
ia/1 
3/1 
0/0- 
0/9 
0/0 
0/1 

*6/3 
2/3 
.6/10 

oA 

2/0 
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As can be seen by scanning Table* 2.1 there -wer^ moVe women ^^amojjng the - 
day-school students than men, more dia'not Have^ a h;lgh-'SchooX dipJLoma than. 
H^d one; the mean age was approxima^lk 46 years and the mean level of education 

attained was about that of the tenth gradfe. On the average, ^tudeats had been 

. ' / ^' ' ' ^ 

in^^this country six years and had, studied , English in their former country and 

in the United States on the average for at)Out one year,. As previously indicated 

^practically all of the students caije ftpm Caribbean and South American countries, 

. • ' . " • 7 

predominantly from Cuba. ^^The.. occupations held by the students in the former 
cQuntry in the United States were mainly lower middle class and lower cla&s 
occupations, though there were a number of students who had had professional 
^ " ^occupntiotis in their former country. - * 

The?-e were also more females than males in- the night-schooi saiJple (recall 
- that in rea^ling the numbers in Table 2.2 that-they ar^p the numbers of students, 
in the six classes that were studied, but the proportions represent the propor- 
^ t Ions in the night-school sample as a whole). As in the day school, a large y 
number .of students did not have a higbt-'School diploma and the iftean age was ^ 
^bout 42 years;'^"he students £[lso had completed on the average 10 years of ^ ' 
schooling; had been in this country about six years and had about a year-and- 
one-half of English bbth in their, former country and in the United States* 
. Again the majority of students came from. Spanish-^speaking countries* » 

.yJE^ day-school and night-school . samples .of students are. similar but 
not identical* There is a higher percental of males in the* night-school 



(44 percent versus 36.. percent) ; fewer students have their high-school diplomas 
(17 percent versus 43 percent) ; and the average age is somewhat younger (42 
versus 45). The night-schpol students have studied English in their former . 



TABl.i; 2.2 



Night S&hool Student Ba' kgrouiiiL Information 
. . Matched Sampler * Descript-^v^ Statistics 



Number ^ 

Sex:"M ^ 
F 

Diploma: Yes 
No 

Age: . Mean 
^ Range 

Education: Hean Years 
Range • * 

Tltije In S.> ^ 
\_-J ' ^ Mean Years 



t 

Frecjuency 


Percent 


Frequency Bv Level 






m. 


' ' J. 






41 \ 








6 


- 18 

23 


44 
56 


« * 

8 
13 


^- V' 

5 


1^- 
5 


\ 7 


17 

83 , 


3 

18 . 


■ 2 • ' 
12 

#• 


2 • 


42 
17-64 • 


/ 


44 , 
17-64 


42 
22-56 


37 
30-45 


to 

3-18 


* 


I 

.7 

3-18 


11 
6-16 


.12 
8-16 



5,9 5.5 . 5.3 . 8.8 



, Range \ 1-15 1-15 1-13 4-13 

^--*..Forocr English * ^ . ' ' 

^ Mearn Ye«ff ^ i.i7 • i.OO 1.08 2^.0 ^ 

\ Range , 0-8 * * ' 0-7 Q-8 0-5 ^ 

English In U. S. - ' . ' ' 

M^Year^ 1,67 - ' ' - " " i^iy : j.^l 1.75^ 



Range * '0-5 0-3 0-5 Or4 



Foroer Country 

Argantlna 
• Columbia 
Cuba 
4 Ecuador i. 
Guatemala 

Ot^er 



cERIC 



Occupation* 

1; Foreman 

2. <:raftsman 

3r Semi-skilled Worker 

.4/ Laboref 

5« Household* Worker 

6« Personal Service 

7« Fireman/Policeman 

8. Professional 

'9. Technician , . 

10. Farnicr ' 

11*^ Farm Worker , ^ 

. 12. Business Owner 

13; HanafgferVQf f icial 

14. Salesperson ^ 

16* Housewife 

17. ^ Unemployed 

18. Student 



^^Previous counCry/USA « 

^ Sec Appendix A for category definitions. 



'2 


5 




0, 




• 

1 


3 't 


? ^ 7 




0 


• '2 


1 


25 


61 




15 


7. 


3 


4 


10 




3 


1 


0 


2 . . 


• 5 




0. 




• 0 


5 


12 




\ 

3 


1 


1 


















Total 


J[, 


2 


^ 

V 3 






' \ 0/0 


0/0 «^ 


. 0/0 


0/0 ' 






,4/7 


yi 


1/2 


0/1 






"4/24 ' 


1/15. 


3/8 


0/1 






1/2 


0/1 


•0/1 


1/0 , 






0/0 


0/0- 


0/0 


.0/0 


« 




- 3/2 


2/1 • 


P/0, 


1/1 






0/0- . 


"0/0 


0/0. 


0/0 




> 


6/0^ 


. 1/0 


3/0 


[ 2/0 • 






1/0 


1/0 • 


■ 0/0 


0/0 






\ 0/0 ' 


, 0/0 


0/0 


0/0 






0/0 


0/0 


• b/Q ^ 


0/0^ 






0/0 


0/0 


0/0 


0/0* . 






. i/1 


cr/0 


1/1 ^ 


. 0/0 • ' 






X l/.P 


1/0 


0/0 


.0/0 . \ 






7/1 . 


6/0 


1/1 


^•^o/o^v'\ 






1/0 


•1/0 " 


0/0 


0/0 ' 






3/0 

•£> ' ' 


2/0 . 


0/0,, . 


• 1/0 ^? 




bxitty approx^matley six months longer (l,fr7 years versus .\ ^17 years), 
(though* the. inajorlty of studeatS;, were from Cuba (61 per'^nt), there' were, * 
^riot as many" Cubans in the night school as in the day school where Cubans 
were' 90 percent of the sample. More housewives and unemployed persons attended 



// / day-school classes. 



-1 



'The picture that emerges from'^these data is of an atfult iimnigrant population,- 
The ^ge range is' considerable, but the group is on the average'Vmiddle-aged 
group. The average level of education is not VhighV "lost of the .p^rf[)le a 
either housewives or unemployed, and those who are working are by and lar 

■ ■ ■ • • ■ . • ^ • - \ 

working in semi-slcilled occii()ations,. \ 



1 



\ Teacher Characteristiqs ' . 

Both day- aneHrl^ght-school teachers ,fiH?^ out a^^questionnaire ^asking for 




the following information: age; se^ undergraduate institution; undergraduate 



major; ,gr^duate^l,nstitution; graduate' majorXs) ^ numfcej: 
degree (s) completed; number of years teaching; grade 
number of years, teaching^ ESL part-time (night-school 



• .of credits .completed;. 



Iev6l of teaching experiencTe; 
adults); number of years 



^ ^^ 'I 4 ^ . " , - 

teaching 'ESL full-time, to children, to teens, to aduljts; number of ESL deminmrs; 

. ^ , * . ' • ' \ . . 

number of ESL workshops; usefulness of undergraduate experience ; usefulness 

of graduate e:qperience; usefulness -of ESL seminars ancl, workshops; peixieht of 

teaching which Vs aduiolingual > silent wa^, and other; and a statement descHbdng 

' their teaching methods and philosophV:. Table 2.7 present^ thef* summaries of 

this information- ^or the dSy*-scnod^^n<i the night-school samples.. 



■ >i 



4. 
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Age- 

Undergraduate 
, InatUtut-ion 



^Undergraduate 
Major 



GraduateMtork 
Institution 
(1st. flA) 



Graduate Work 
ilajor , . 



-TApl.rf 2.7 



Teacher Background Informajtion 



DAY SCHOOL (N =..6)' 



35 



Range: 24-45* 



5' Females;' 1 Male\ . . 

** * 

Jersey City State; .2, 
"Douglas: 1 
tadycliff: 1 ' 
Montclair State: 1 
Seton Hall: 1, 

Er^mentary' Ed. : 2 : 
Spanish/Ed. : 2 
Phildsophy: 1 
Italian: h 

Jersey City Stat?:* 2 
Fairleigh Dickinson: 1 / 
Montclair State:, 1^ 
Rutgers: I* 
Seton Hall: 1 

Elementary Ed. : 1 - 
^ESL: .1 . ' 

ESL and Ed.*: 1 
ESL an4- Reading; 2 
Guidar\c^ and P^sonnejl: 1 



^Number of Credits^ ^ N 6;^X: 30, Range: 9-47 
'Completed ' ' . *. , ' ' • 



% 

L 



^ , 1st Master's 

Degree Completed 

» 

Graddate Woric 
. ^fistitution ^ 
• (2nd MA) ^ ' 

V Graduate Work' 
- Major 



NVimber of Credits 



N = 3 



Rutgers: « 1 



Italian Lang. &-Lit.^: 1* 
N = l,'vX":''27 ; ' 



■ . NIPHT^SCIIOOL (N = (?) . 
X: 35 ,__^_^i1r,e : . 23-5^5^ 

3 females ^ 3 blalies /. 

•ier&ey City.. Stat\^: 3^. 

Missouri: 4 
Saiat Peter's: 1 
St. Elizabeth 's>^. J. " 



El;^mentary Ed. : -4 
HistoT^:' 2 , 



Jersey City State;. .^' O 



Elementary^ Ed. 
Reading: 2 



= 3, X.:^v,32„ Range: 30-34 
N ^ 3- ^ * 



Jersey City Stat^: . 1 
Montclair, State: 1 
Fairleigh Dickins.oa: ' L 



•U 



Eleraeri'faj^jr' Ed.: 1 



' \ Coinpieted, 

\nd" Master's Degree - N = 0 ■„ 
• -/ djinplecefd " „ 

Q ■ T\he N for eaqh category '^-e" unless otherwise stated. 



N = -3, X: 26, .Range" 15-3^^i;"'v 



>^ 2 . 



CV^/Nuraber ol Yfeajrs v 
> Teaahlng 

? • Teaching 

pr'' Experience l^vel 



1% Years^ ESL , 

at Night, Adults 

Yeajs ESL Children 
Full Time 

' Years ESL Teens^ 
Vull Time., . 

Years ESL^Adult^ 
. Full Time- 

1.,^ iliimber of ESL 
Seminars 

\ Number ^of ESL 
' Wprkshops' * 

. Usefulness 
^. ^ ^Underg:^aduate 
r/'/ Expe^ience*^* 

: Usefulness of 
: Gra'Suate " - 
|f/^*^Experience** - ^ 
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TABLE 2.7^ (Continued) 

DAY school '^ 

Range:- 1-18^ 



Elementary; 3 
High Sc^hooi:. 1 
Adults: 2 ' 

' .4' 

» N = 2> .X: 8, Range: 3-12 



N = 1, X.: 4 ' 
N = 0 

N = 6,. X: -^2) Range 1-4 , 
X: .4, Range: 0-8 
X; 17, Hange: Ac^O 



-J X: ■ 4,, Range: '2-5 



V 



' NIGHT SCHOOL . . - 
X: 10^ Range: 1-^26 



•Elementary:' 4 
Junior High: 2 



N 3, X: *4, Range: 1-7 
N =^ 2,- X: 4, Range: 3-4 



N'= 0 



X: l,^Range: O-Sp 
X: 13; Ranger 2-25 



X:. 3, Ilange: 1-^ . ^ 



r= 6, X: 2f, Ranggj: 1-4 ,, N. = 3;^: 2, Range: 1-3 . / 



-iC: 1, Range: - ^-2 



|^|Ose|ulhe^s, of 
|^?^ESL Seminars 
l^fiaM* Workshops 

1;^:-^; Percent Audio- 

IC^'iinjgual.IMethoii . . 

' % ^ . - - . ^ ' - i ' ^' 

%^Bercent Silent- 66%, Ranee: ,30-99^^"^ ' 

Method . : / ^ f ' ^ ...^ 



X: 1, Range:. /.I * ^ 



N - 2, X: 62%, R^nge: 60-6-5% N ='6,vX: 00%, Range:740r.99f 



X,: 35%, .-Range: 10-60% . 



E^rcent^ Other 
P^'Methods. 



N = 5, X:^ .14%, Ranj^e: 1-30%; = 1, X: 30% 



^;:':riii w'£ox. each- category =.4 unless "otherwise stated. 
fc^^Verv useful^ 1; rior!'at-ail useful 5.'- ' 



4^. 



' ^ \ . - • ' 



« 
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TABLE- 2.7 (Continued) ' , 

DAY scho.ol'^^ iuniiT ^^rnopi/ 



N - 2 






4 


N ^ 1 






0 


N = 0 




*k = 


1 


N = 1 




N = 


0 


N = 3, 




N = 


0 


N,= 3 ' 




: N ^ 


2 


N = 0 






] 


N ' = 1 




N = 


0 


N 1 


• V 


N ^ 


1 


N » 0 ' 




N =• 


1 


N =;o 




N = 


2 


N-= 1 ^ 




N = 


1 


N = 4 




N' = 


4 


N = 3 






3 


N = 4 




N = 


0 


N*= 2. 




• = 


1 


N = 2 




• N = 


1 



ft 



,t ' t 

6 unless otherwise, .stated. , 



The^ two samples differ in several ways. Th^i majority of day-school 
teachers' were^ women while only half the night-school teac+iers weret All - 
sljTdayrtschool teachers had completed, some graduate work, three had completed** 
a Master's degree and one was working on a second djegree; while only three 
of the night-school teachers had done graduate work — all three had completed 
a Masyter's degree and two tf th^ had completed a second Master'-s degree* . 
Night-school teachers on the average have had more experience in teaching, 
but this apparent difference is due .primarily to one teacher who ha^ taught 
for 26 ydars. Most of th^ day-school jteachers' experience .with ESL had been 
teaching adults; whereas,^. most of the night-school teachers also t:aught ESL 
to children or teens during the day as well as to adults in >the Evening* 

The te'achers' statements about their teaching methods reveal differences 
between the two groups. Only two day-school teachers said they used the 

v^.. ^ • ^ h 

audioiinguai method of teackiiig and estimated that 60-65 percent of- their ^ 
teathing was of#th±s kind-., . But all of the night-school, teachers stated that 
they used the audiolingual methods and their estimates of use ranged from 
,10-90 percent. The two groups differed ^.it^ philosophy in two respects: day- 
school "teachers more frequently stated that tijey believed that teaching' styles 
should be eclectic and that, students should dominate classroom interaction. 



THE MEASURES OF STUDENT PROFICIENCY ' IN ^ENGLISH 



The measures of prof iciencX- ^n' English were ddminis*ered at two^diff erent 
times:" for day school in November, and again dyring A^sil and May; for 
♦the night school in Ma^ch and early April and again iA June. The reason for 
two adininistxations was to estlnjate the change in proficiency Jjetwfeeh the two 



testing times. The first test measured how proficient; the students were .neTar . 
the beginning of the instruction that th^y were receiving that year or • 
semester;- the second administration measure^ how proficient they had become 
after six months of instructiori (day school) or three months (night school). 
Test^.dates and procedures are discussed in Chapter II, . , 



The Oral Proficiency Test . , 




The major goal of Che Center is to increase proficiency in speaking 
English. Therefore, the most appropriate measure of the effects of instruction 
would be a measure of the studenf^s ability to speak. English. After, a search' 
fpr and examination of available .procedures we concluded that none of the 
procedures that ve had examined met the 'requirements of a .standardized testing 

procedure. Consequently, we developed an Oral Proficiency Test, 

^ ' * > ■ * ' 

The first step in developing the Oral Proficiency Test was to gather 
from the teachers in the Center a list of objectives for each of their classes. 
Each teacher provided 20 objectives and across the set of day-school teachers 
a composite, list ^ of 132 objedtiv^ resulted, The^ 132 objectives' were then 
given to the day-school teachers who rated each%5>ective on (1\ whether or 
not it wa? something they taught; (2), how important they, felt it Was for ^ 
proficiency; (3) how much empha^l^ they gave it in their classes; and (4) how 
difficult they thought it was to learn. These objectives and ratings were then 
used to develop the crfntentof the Oral Proficiency Test. 

The test ^Iself was an individually administered test that Required the 
student to speak'in English. It had three kin'ds bf items. The first set of 



items were based on representative examples of/ the teachers^ objectives, for 
each leveLof proficiency. These items were in a structur^ed, conversational 
format and arranged in order of difficulty by objective. The'second type of 
item involved presen'^ing the student ylth action* pictures-; the student was 
asked to 'describe what was 06curring in the picture. Both types of items 
tested the student's ability ta generate language freely. The third type 
of item consisted of three Spanish cartoons and the student was asked to * 
state in English what was occurring in the cartoons, The purpose of this 
t$T)e* of item was to see if the student could go from idioinatic Spanish to 
idiomatic English. The test was .piloted and then revised >Qn cl^e basis of 
this experience. 

The tests were administered individually by trained testers. A' recording 
was made of the testing. The test itself was administered to the students near 
the end tff the school ^ear (it was not used as a pretest).^ The test yielded * 
three kinds of score's. One score was for comprehen3ion, meaning that the. 

' ■ • ■ . ■ ■ • "T' ■ 

student gave evidence of understanding the questions asked of him or her but 
could not produce a correct English response. The second score was given for 
selecting the appropriate^ structure to use in a response even though the 
student made oth^r errors in^ responding. The third score vas fop correctness 

■ - j ' ■ • ' . . ."• ' • ■ 

as well as*^use of appropriate structures. Thus the first' score represents a- 

V • ' * . ' ^ ' ' ^ 

.level of proficiency in which the student appears to be understanding the 

English spoken* to him or her'^t responds inadecjuately*. The second score 

represents a higher level* of proficiency^ on'e in which the student can respond^ 

and does -so by choosing correct structfures but whose response Is not entirely^ . 

correct. The third score replresents the highest level of proficiency in which 

/the student responds in an entirely appropriate and correct maniil^^^ ^ , ^ 



^ ^ * . ^ The Literacy Test ^ . ' ' 

A pot6nt^l qide effect of learning to speak English is that ^studeijts 
may also learn to readmit. They are exposed Outside of the school to English 
newspapers in Englisii, directional signs i,n English, and a variety of .other 
printed matter-in the English language. As they acquire words ±a the language 
and .understand how to speak the language, they may al3o acquire the ability 
to read it. Furthermore, during the teaching of English as a second language 
wo^ds are written on the boaijd and printed materials are given to -the students 
as a- basis for conducting conversations. « - 

Therefore, we used a Literacy Test which m*easures the functional tfeading 
level of the student. - The materials irf the test consist. of pictures of signs, 

* labels on bo'ttjes, forms, tables and so on. 

There ^Were 50 such items in the LiteragjtsrTest adapted from an original 
siet' of 170 items which had been d^^^^floped^ to measure functional literacy of 
adults who spoke English.*^ The items selected, for thie study were pretested 
•xd.th two students from the Center, one of whom had little or no English 
profi'ciency and an advanced student with considerable proficiency. After 
tbi&^-pilot testing,, the test wa« revised and shortened. 

Instructions for ^the test were ^given in either Spanish or English depending 

upon fhe students' level of proficiency. The queation for each item was read 

• * . • ^ . * ' ' ' ^ .* * , , ' 

to all students in English. They then read th*e item and indicated' the answer 
''to the question by circling the appropriate word or^^ntence. Testing was 
stopped when the students answered f iye^ consecutive items incorr(BCtly. (The 
itepas we^e arranged in order pf^tJieir- difficulty for ^English-speaking adults.) 
Thfe student score was' the number of correct answers; ' < 



j,^, f ^ ^t^Murphy, R» T. AdiUt Functional Reading Study.' Educational "Testing Service, 
; New Jersey, 



" * ' Oral .Decoding Test 

^ We thought that stifdents might also acquire decoding skills as a result 
of learning to speak English. Obviobsly if students acquire general' skills, ' 
such a9-decoding, their language proficiency shoud increase rapidly, and 
they can' 'become Independent learners of vords and structures. ' We administered 
an Oral Decoding T^^st^^veloped originally to measure the . decoding, skills of 4 
young children. -Forms of -this test at two difficulty levels were tried out 
on the same two students who ,had taken the,lit^Ar.y items' and the results 

■ . , . • r ■ \ 

suggested that the more difficult test shouS^ be us_ed.\^ ' 

Instructions were modified so that the item stem was read to the student 
rather than having the student read it to himself or herself. Instructions 
were preparefi^both in English and Spanis^T^owever , test items wiere r^ad onjy 
in ^English. This tefet was also administered as ,a pretest and posttest? The 
score on the test was the number of "correct answers. ^ 



Other Measures of Proficiency ' a> ■ . 

Two other .mefastires of .proficiency were .used as pretests, the John Test 
and the Morano Test. Thfe. Center had been using both of these tests to estimate, 
students^ initial 'prof iciency in order tp-place them in classes. ; , 

John Test • . * ^ • * . • 

This Oral Proficiency Test (developed by Linda Kunz of N Hunter College) 
consists of eight pictures a'bout which the student *was asked 22 .questions*. 
The student, is given a comprehension^^score ^and is rated by the\ tester on 
fluency,, use of structures, pronunciation ^and vocaibulary. ' \ ^ 



Deyelpped by RoyCrt and- Kathryn Calfee, Stanford University, for the Beginning 
.Te4<iher Evaluation Study conducted for the California Commission for Teacher 
Preparation and Licencing by Educational Testing Service, Frederick J.^ 
McDdii^ldj 'Proje'crtv Director. , , ' ' ' ! 
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Morano Test ' i 

'*^'The Morano Test is a paper-and-pencil test of recQguitioh of correct 

use pf English grammar. It hafe 50 items each consisting of three'^sentences 

~ expressing the same idea but only one of which is grannnatiea,y.y correct. The \^ 
student is instructed to read the items and indicate the corre^t^ sentence. 
This • test «was also administered as a.pyetest. 

Thus there are five kinds of tests. all of which haVe face validity ^.for 

^measuring' proficiency in English. But of the five tests used, only two are 
direct measures of pToficiency in speaking English, the Oral Proficiency Test 
and' the John Test. ,The difference betw-en th6se latter two tests is that the 
Oral Proficiency Test samples some different aspects of speaking English such 
as free production of language. "^Two of the other tests, the Morano'- and* the 
Oral Decoding Test measure skills, presumably related to proficiency in^ spiking 
English.. *Trhe Literacy Test measures proficiency in reading English at the 
functional LeveJ. Proficiency on this^te^t vould be regarded as a Heneficial 
side ef fect in the perspective of this study. ^^^^^^^ • ' 



V 



Student Performance on the Measures o5 Proficiency 



•.Table 2.3 presents the. data or' both s^amples of students, day school •and 
night school, on these^^measures. For, each test information is giverC on the 
number of -Students taking the test, theiy .mean score, the range of scores, 
the standard deyiatioi; of th^ scores, an^ the ueliability of the test. As 
can be seen by examinitig' Table 2.3, all the tests, have high reliability, the 
lowest reliability' being in the -. 70-^-3 for.^the Oral Decoding Test. 



%. ■ ... ■ . ; . 340- ■ 
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tIble 


2.3 


• 


- 




• 

• 




STUDENT TESTS: DESCRIPTIVE STATISTIGS 


— ■ 








When-' 


•N ' 


•* 


■Range 




Reliability 
















Day 

' Day 


• November (Pre) 
April (Post) 


148 
ll9 ' 


29] 82""- . 
37.91 


'■ 3-49 " 
11-49 ■ 


■ ^'l2.52. 
7.22 


. 9,6 _ 


' Night 
Night- 


March (Pre) ' 
Mav fPost r 


45 • 


34.73 
36.81' 


11-4 8\ 


'. 9.70 
9.98 


.93 
; .94 * 


Day 
; Day 


A'pril (Post) 


1.^8- ■• 

no 


26,72 
28.42 


13-37 


7^0 
■ 5.33 


.88 
■ -.81 


Night 

Day' 
\ Day 


March (Pre) 

November {tte) 
April (Post) 


148 

izu 


29.4.7 
^7.84 


14-37 

31-57 
36-59 


6 90 
. 5\ 28 


.88 •' 
.74 


^ Night , 


s ^ 

March (Pre) ^ 


45 j 


'49.33 


Vf 59 ' 


^ ^ • 00 


73 




November (Pre) 
April (Post) ' 


148 
120 


80.88 


49-93 
56-95' 


11.03 
• 7,90 


.89 
.83 . 


Night 


: March (Pre) 


45 • 


78.80 


60-96 
f ' 


• 9.47 




Day ^ 
Night 


. June (Post) . . 
June (Post) 


X 

itr , 

43 ' 


30.48 . 
27.65 . 


2- :58 

3- 53- 


♦ 

14.66 
•16.21 


.96, *- 
.97 



Day 
Night 

Day 
Ni'ght 

Day > 
Night: 

Day . '^ 
' Night 

- Day: , 
Night 



' June.; (Post) 113 

June (Posb) . ' 43 

f ^ - • * . 

' June^ (Post) • • ' 113 
'june (Post) * 43 

: June (Post)^ ^ 113^. 
June (Post) * ' '43 ^ 

NoVember^CPre) X15 ' 
"March (Pre)* . - -46 

November ^Pre) ^M^^' 
' - 'March (Pre) . • 



.11.^ 
lf).49 
i 

14.»53- 

12.69*"* 



9.31 
7.70 

37.27 
30.54, 

26:82 
26. 3P 



.0-45 

-0-41 
0-33 

e-26 

■ 0-1? 

. 0-70 
^ 0-65 

3-50 
.7-46 



. 8.84 

10.16 
10. Ql 

4.86 

k.ni : 

20.54** 
21.07- 

11.47 , 
10.46 



..92. 
.92 

.93 
.94^, 

.72' 

— 71 1 

-.87 ' 
.86 ■. 

'.93 
'.92 



241 



* Table 2..5> presents two kinds or inforttation; in the upper half of the 
'table are the inter correlations amoi^ the tests; in the lower tialf of the table 



are tha correlations of each test with the .laackgroundC^gligfracteristips of tKe ^ ^ ^ ^^^^^.^ ^^ ^^^ y,.. 
students. '(Nd'te that the letter P means fall adminisli^ion of the.tegt, and 
the letter S means spring, administrattpn of the test.) \ , • > 

As can be seen by exainining Table, 2. 5 "the tests are all highly intercorrelated 
with' the' exception of Decoding. 2 which is af^ptire measHre of decoding skillT^ 
^Decoding 1 measures word attack skills^ and vmderstaixcling of xfc rd • jmean in gs, and 
is an estimate of general language proficiency. It is not surprising that 
Decoding 1 -,'cqrreiates with other measures ^f prof iciei^'cy; 

As can also be seen \vi tliat table tlie three scores of the Oral Proficiency 

* * • , 9 . ' ' - 

* Test are highly qj)rrelated, so that it is reasonable' to assume that the Oral 

^ 0 *. _ * * 

• f <i ' ' » ./^ ' * 

, Proficiency Test is measuring ^ general .factdr% It is also interesting tcb. 



note that the Proficiency Test comprehension subscore hasNa high correlation 

with the"' John T^st Vir'*- .85). As we suggested above, both of\hese tests are 

- * ' i ** * **** H 

measures of speaking ^rpf icienqy* , 




-I 



The lower half of the VajSle presents the, correlations among- the backgroxind 

• factors and the test scores for ""the 'day^^dhool /satopjjg.'. Very fewiof these ^ 

■ - • , / ■ • ^Jfe • *■ • 

correlations are substantial' in magnitude ,,^ahd. only'those. equal to or larger 

an .30 are statistically signif icantfy«.JPfier6 ajae a. number of interesting 

relations in this correlation matrix, however^. Tl^ere is no relation between 

' .sex of student and tjest scores, -but there is aftnegatiV§ correlation -of jige : 

' - ' r (. / 

with all of thj ^st scores. Considering only thofee whdLch 'are statistically 

-■' *, ^. ' " ^ * * - ' 

significant, younger students do better\on .^He Decoding, Literacy and Oral ^ 

Proficiency Tests. Neither time spent in the U. S. or having a diplomats 

. related to>theufail Literacy^est acot^e^ *Note, however^ the siftfet^n trial 

correlations between years of education and all , of the test scores.* 



.lABM- 2.5 

Intercorrelatiorts of Fall and Spriaf* Tesfes^and 
Student Backgroui;d Information 
Da> School— Matched Samrlo (N ^1) 





D -F . 

1 


£. 


l.-F 




41-F 


' Dercadingj^ - F 










• 


: Decoding^- - F . 


. .50^ 


1- 








, Literacy - F , 


" .-65 


.41, 














.77 






^ohn — ^ F 


^ : 56 


.29 , 






Morano - F 


.55 




.59 


.78 




Decodingj^. - S 


■ ".73 


.40' 


./.8 


.42 


.45 


" Decoding^- - S 


^33 


.11 


.22 


.,10» 


.15 


J Literacy^ S 




.31 


• .7i ~ 


, i64 


.;51 


Proficiency 












Correct 


.60 


.26 


^59 


.68 


;.'.63 




p-CT p~n^ 



P-ST. 



.41 
.47 



' Comprehension' . 70. 
Structure .63 



.39 

• * 



\30 



.75 

:g3 



Sex 
Age' ' 
.Time in U. S. 

•iiiploraa- ,,* 

.... . ^ 

VFo«iner Job 



..07 ;10 



.17 



-.3-2- -.JL5 -.30 
-.15. , -.05 ' 



fcaob in 'U. S.. 



';C,-Orl-jgin- ' • 
iY,ears E^trcar.tidn 
, Former^ Eng; , 



•-21. 
•25. 

.05 
■-.'21 
'-.16 
".28 



.08 
-.12 
.12. 
.19 



.L8 
.22 
^39 
.21 
.02 
.33. 
.25 



£fvg. in U. -^S',^ 

|pr< "^'05 ;f &r r > .22 
lp#* .'QI for' r >- .28 ' 



.13 '#.^41 



85 . .73 . 
74 .68 



.5.1 
.63 
.54 



sd6 

.22 .47' 

' . ^1 .61 

423' .51 



1-86 
.97- 



..90 



17' 
13 



.02 
.05 



13 -.07 

t 

11 .-r09 



33 .'33 
20 . . 19'. 
07 -.15 
18 .16 
26 . . 17 
51 .h 



2.43- 



.03 


-.06" 


'-.02 


. .12- 


'.19 


.U, 


-.27 


-.06 


-.28 


-.18 


.-.23 


-.'21 


-.13 


. -'. 08 


■ .00. 


.05 ' 


- .06 


.04 • 


.22 




!0.9 • 


.12 ■• 


..18 


.16 


».29 . 


,18 


..-22 


■ -.24 


.32 


.27 


-.26 




.24 


.13 
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PEVELOPMENT OF THE 'OBSERVATION SYSTEM . 

- «* ' * • . • ^ 

« 

* Teachers* ^ethods, materials and interaction with students were coded 
with an in-cla^ observation' system. This system was based on what^ the ESL 
teachers and students at the Center actually do. To develop the system, the • 

^ * . " > \ . 

ETS staff obse^rved in several classrSoms and then discussed their observations 

4 

1 

with the coordinating teacher* -The staff spent much of the first three weeks 
of the project observing in Center classrooaiB and talkiruft with the teachers. 
Th6 last ^e^k i<f October each teacher was videotaped t€iJ[:hing a thirty-minute * 



cla§s. 

On the basis of th^e in-class observations, the videotape's and discussions 

■ - ■ • ■ > ■ - ' , . - 

with the project -director and coordinating teacher, we developed a' preliminary 
fcyfrn^c^f the observation system. This first system was tried cjquit in ^he Center 
^classrooms. Repeated .modifications were made and a lexicon o^^def iiMtions . ^ 
prepared. During this tiwe ac^tual observations were discuss ed to clajr^fy 
definitions of--categories; categories were expanded or eliminated; videotaped 
and in-class examples were considered in developing ,the final catego^ries. An 
important consideration in developing the system was to make sure that the 
categories, would describe the differences between the /two major teaching - 
Strategies used in the Center. The final draft version was us erf in preliminary 

observations "of flight-school classes as well as day-school classes^ to insure 

i . * ; ' • . - - . \ 

Its appropriateness to both settings. . * 

' ' ■ ■ /- ' 

Description of the Observation System 

The result pf the development work described above j^/as a categorical 
' * observation system which allowed for sequential coding of classroom behavior 

ERIC * . . / ' . . i''21i_ •: ^-f - •. 



A reduced copy of the Observation Cdding/Sheet: is shown ^ in Figure 3.X» A, 

^complete lexicon and a' sample observation are in Ai^pendix C# " 

V .. Three superordinate categories^-con^^^xT^^ns^^^^ design), materials, 

^d strategy (method of instruction) — describe the classroom setting within 

which the teacher and student behaviors are recorded, -Each of thes§ categories 

is subdivided; e.g. , tKe context c^n be drill, writing, explanation, dictation, 

etc*, and each subdivision has a numerical. Code, These (Categories are code4' 

initially and recorded again only if they change during the^ observation period. 

' / » • *' 

; The first group of teacher beh^ior categories — questions, serial ^r^irects 

. \ - " ' ^ ' ' 4b'"^ ^''^ 

direction, yodels, .writes on board, explanation, and Dther-.-are discrete. 

instructional behaviors which, uaually initiate, 'a teachex-student (s) interaction.. 

Where these behaviors are carried out nonvefbally, they are* coded^with an "N" 

rather than a check mark. \ The next column — -'class, group, individual— •signifie^ 

to whom this behavior, is directed. . ' ^ ^ ^ * 

The fi^^st gyoup of student behavior categories — answers, free response, 

3.^ 



practice, ^rites oit 'board, rea^s, chooses, not to respond, asks question, 
(particip\^s in) conversation, student-to-student feedback, and other--are , 
thd^e, behaviors which either follow the teach^'s initial behavior or initiate 
,an interaction on the part of the student* Conversation and student feedback^ 
are coded- with an "S*' if they dccur *ln Spanish rather than English. 

The next* three categories — positive, corrective, negative — describe the 
possibj-e types of teacher feedback. Here again, since nonverbal feedback is 
an important component\of "silent .way" instruction, an "N" used to indicate 
a nonverbal res|ionse\ 



FIGURE "3.1 



r. .qt>servation ^^Je Sheet *with an Illustration of a Pcfssible Classroom. Behavior Sequence 
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The ^second group of teacher, behavM^ categories— models, prompts, asks 
"to rep^^- repeats, explanation, writes on board, dljrection, question, an(i 
other— designate response behaviors ©n the part of^the teacher. Again, they 



are'' coded to Indicate whether they are given verbally oir nonverbally. 

The secpnd group of student*i)ehavior^categories— student 'models, student 

prompts (these two usually follow a direction frbm the' teacher) , answers, 
, free/ response, practice, writes on board, rea^s, chooses^'not to respond,' 

ask^*question, (participates in) conversation, . student-to-student feedback 

and other—describes those student t)ehaviors given in response to the teacher *s 
^ response to the student's initial behavior* or response. Again, an *'S" is 
^used ta code thos# responses which wejre given in Spanish. ^ • 

.The commeat column allows the observer to indicate what the "other" 

behaVioT'codea^birTliat^ is or to note iome unusual clasgiroom occurrence. 



yse of' the Observs^t^on- System ^ 



, The .observation system .was" designed fot in-class use. When the observer 
first entered the room, he or sha^ook a, few minutes to tiij ou^., the tbp of 
U^^iigaifi^ijservation sheet, " indicating the date, teacher, time, number df students. 



obserx^er, class^et-(ip\ifagram) and language structure being taught. During 
:ih£S orientation time it was possible to code the three superordinate categories, 
and then quickiLy proceed to , accurate, coding of the classro&m interaction behavior 
(See Chapter Three for an exampleiof ihe coding.)'- , . 



Ob^ierver Training 



Usltig the videotapes of the Center teachers, the cpo,|:dipa£ing teacher and- 
* ' . , ^ ' . , ' • ■',"■•,;:-_,.* 

Im. • . Both- testerrobservers were traia<?d on t;he system. In-class training was 
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accomplished by .placing two observers in the same classroom' and comparing 
^ their intera«t±encodes fcM^agreements and disagreements: Thosefcategories 
where the highest disagreement occyrred w^re in several cases further modified 
and in others better clarified; tp improve inter-observef agreement. Oribe,'' 
these final modifications were completed, reliability studies Vere' begun. 

y} i)bsQrvatlon Schedule 



/ D^y School^ • ' ^ 

■ ■ • ^ . / ■ 

A schedule for observations was prepared which allowed for a 20-minute 

' \»- . : ° ' ■ ■ ' ' ^ 

observation of each c^ass on four different days of the week. , For those 

^- classes meeting only two or three times a week, observations were scheduled 

for each meeting, (See the latter portion of this chapter on the reliability ^ 

of the observations^" r Observation of the day-school classes, began January .19th' 

. " . ^ " ' ' • ' ' - ^ *v 

S • and continued through Aprfi 1, 1976. , V ; . _ . 

Sevei;al factors affected the actual number of observations made: e.e. 

-school holidays and xl9Sings as^ well as\teacher and observer illness. Thq - 
■ « . ^ ' ' \ 

number of observations per class and' observer are presented in Table 3.1. 

^ / J ' / . . 

Note that 'for all but- three of TThe classes (two of which met only two or 

three times-a-week) , we had an average of 30 observation sessiptls. per ctass, 

g-iving us a . data base of sotae 600 minutes of observ^ation per class. 



/ Night School * : ^ . ' 

A simixar observation schedule was drawn up f6r the night scfhool. Here^ 
however, ^sinqe class meetings were held for two hours, three times-at^week, one 
observer observed during' the first half of a clafs'and a second observer 
obsexrve(d during the second half^ Night-school oT>sefvationfe^ were made by the 



-< TABLE 3.1- 



Number o'f Clcipsroom Observacidns : By Clas§ and Obs«i;ver 



Day • , 
Classes 

0221 
0233 . ' 

0313 
'0323 

0412 ^' 
■ 0422 
• 0433 
.0511 
. ,0522 
^^621^ •• 
■ 0711 

0721 

0731 
iOTAL - 

r ' 4 

. Night 
Classes , 

. ' 0901^. . 
^ 12(^3 

^ 3402/ 
' 1601 
',•,1701 - 

, 2i02 

TOTAl. 



0 



Observer 1 

8 • • 
.9 ' 

1 
15 

/ 15 
. 15 
17 
■ 16 

ibr 

15 , 
^ 15 

16 ' 

- 

16 



5 
7_ 
5 
5 
7 
• 4 
33 



Observer 2 

0 
0 

1 ^ 

• - -.• 0 

- 

. 0 

2 

n 

0 

.1 
i 

< 

6 



3 
• 2 

■4 

'^1 
3 

_1 
14 



Observer 3 

21 
6 

16 ■ 
1 

■ 44 
^5 
13 • .' 

15 ■ 
15 ~ 
13- 
17 
■ -14 . . 
14- . 
1 _14 
188 



' 4 
2 
5 
4 

! 4 

4 

h 



\ : . 

• .29 

\l5 

1-8° 

16 , 

29 
. 30 

32 

31 

31 

29 

3^~ 

31 
' - 28 
_30 
■ 382 



12 

ii 

14 

'10 
14 
9 
70 



..same tester-observers and the coordinating teacher. Observations were made 
at two' different times during the semester; between -February 5th and March 17th, 
1976, and again between May 3rd and May 25th, 1976. "\ . ' ' . 

. ^ Again, the niinber of observations per class and observer are listed in 
^ Table 3.1. For thi, six nightrschool classes we had an average of 12 obs^ervations 
, P^'^ e|0s,f-=giving us a data .base of ajiproximately 200 minutes per class. 

. " ■ , V Processing of "Observation Data , 

As class-room observation data were"being coliected,4ke data were being ^ 

transferred and C9ded onto forms which were^set to ETS. *These data were processe 

and data sets were made for us,e ^s-ihput to 'the comfjuter. An editing procedure 

similar to^the one used 'for student, data was used/on these^ata.' ^ • 

. Data for analyses were then created and this process is -described in 

> ' ^ . ■ ^ . ■ 
the section entitled, "Analysii of Behavioral Observations;'? Many analyses 

including summary statistical, 'correlatioris and fact^,r'airalpes were completed. - 

Factor -scores were then created for each teacher and were placed on each 

student's record (the appropriate student and" teacher were matched) for the 

^ final analyses. Appendix D contains the details of the data processing.- 

^ • Des^criptiYe Statistics Pertaining to the Observational Data 

Table 3.? gives the "observat'^n 'item numbers, ^tegory labels*, .codes and .. 
means- for. day and night-school classes'. The category means may be interpreted 
-as the_Erbporeion>f - observation episodes"^ in wljich the event was observed- - • 
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\n episcTde is d\^f ined as . a sequence., of .behaviors betw^ the teacher and ' 
.;any particular st\ident.- It m!ay be ^Initiated by either teacher or student 
/and; ends \?hen the' teacher a'ddre.sses or responds to another student. 



J. 



.if. *. 




• * • 


Number of 
Categories > 


X Category Labels 


Codes 


Day School ^ 
Mean&+ 


Night School 
Means+ 






1 Context^. 


2 


1 Drill. 
2 


DRILL 
OTHR 


• 99 " 
•Ol 


<• 

1.00 . 
.00 



1 .Materials 



Uo Materials 
Rods ^ 
Cartoons 
Pictures 1 
Mir.co Sheets 
Objects 
Sight Words 



NO Mi\TLS 
jRODS 
CARTOONS 
PICTURES 
MIMEO 
OBJECTS'- 
SIGHTWOS' 



.53 

Ai . 

.02 
.05 
.23 
• 04 
.02 



• 65 
^.12 

.00 
.05 
.17 

• 00 
.00 



'3 Strategy /Model 



1 Question Answers" 

2 Free ^Response 

3 Repetition 

4 'Directed Dialogue 

5 Discussion 



Q&A 

FREE RES 
REPEAT • 
DIRECTED 
DISCUSSN 



• 94 
^•03 

• 00 
^ .02 

• 01 



.99 
*-01 
^00 
.00' 
• 00 




4 Initial Teacher Behavior 

5 Initial Teacher Behavior 

6 Initial Teacher Beluivlor 

7 Initial Teacher Behavior 



2 


2 


Question 






TBl-QUES • 














2 


, "2 


Models - 






TBl-MODC* 














2 


. 2 


Writes on Board 






TBl-WQB 














4 


. 1 


Serial .Redirect^ 






TBl-SR 




'2 


Direction 






foi-DIR 




' 3 


^ Explanat>lon 






,TR1-EXP> 




4 


Other ' 






, . TBl-OTHR 


.. .. c. 




.1 - ' - 









-32 

• 10 
.05 

.01 

• 18, 

• 04 

• 03 



• 40 

• 11 
.06 

• 02 

• •18- 

-.09 
-•01 



8^^hjecc Jbf Teacher Behavior 



Class I 
Croup I 
Individual 



CLiNSS ' 

GROup^. 

. INDIVOL 



.01 , 

■ .^0 



.31 
• 00 
• >69 



. , / Initial Student Behavior 

/ 10 jinltlal Studfetit Behavior 
r. • 11 . Iniclai SC^ijent ^B'e^^ 



/ - ' . * 
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2 Atiswer^ 

2 Practices 

1^ Free pe^pd^nse 

2 WrtCtfs on J5oar4 

3 'Read>s ^ 

4 CJjooi^os Not -to Respond / 

5 Question 

6 Co.nveraition 

7 StudenC-Si^udcnt Fee<lba*ck 

8 Other ~ . y [ 



f 



d / ^<Mt^^ 



Sffl-ANSR 

SBl-PRAC 

,Sb1.-?FR£E 
/SBIVWOB 
SBlrrRHAD 
/ Spl^CNOT 
' ' SBl-AQ 
.SBl-rGOllV* 
SBl-SFBK . 
: SBl-OTfiR 




Jltera 

12 Corrective Feedback 

13 Qualiey of Feeback 



Number of 
CategSbies 

* 

2 

\ 4 - 



Categgry Labels 
X Corrective 

2 Positive > 

3 Negative • ^ \ 
^ Other 



Codes^ 

FB-tCORR 

FB-OiC 
FB-POS 
FB-NEG 
FB-OTHR , 



Day 
School 
Means+ 

f« 7- 



♦ 50 

.07 

• 05 
.01 

.00 



Night 
School 
' Means-f 

.30. 

.02 
.06 

• 01 , 
.00 



U 'Successive teacher Behavior ✓ 

15 Successive Teacher Behavior' 

16 Successive Teacher Behavior 



2 
7 



17 Successive Teacher Behavipr 



2^Modcl1^ , 

2 Prompts * 

1 Asks Student to Repeat 

2 Repeats 

3 xE^cplanatiori' 

^ Writes on Board 

5 Direction • * 

6 Alternate Response 

7 -Other 

2 Question 



TB2-Monr. 



Ta2-PR0M 

TB2-ATR 

TB2-REP ■ 

TB2-EXP 

TB2-W0fi 

TB2-Dlk 

TB2-i\Lr 

TB2-WIR 

TB2-qUES^ 



.22 

V- 

.16 

.0^ 

.07 

.0^ 

.02- 

.02 

.01 

.09 



.22 

.20 

.07 

.07 

.07 

.09 

.02 

.00 • 

.00 ^ 

.12 



18 Other Student Behavior ^ 

,19 Other Student Behavior 

20 Successive Student Behavior^ 

21 Successive StCxdent Behavior 

22 Successive Student BeHavior 

■ .1 ': . ^ 



2 . 
2* 



2 Student Models ' 
2 Student Prompts 



2 Answers _ 
^ / • 

1 Writes on Soard 

• 1 Free Response 
X ^ Prnc rices 

^ 3 -Rends . 
, k Chooses hot»to Respond 
5 Askfe Ouestfon 
6 ' 0>nver«a?iort ^ ^ 
1 Studfent-Student Feedback' 
8 • Other " k " . i 



SB2..SM0DL 
SBZ^SPROM 

SP2-ANSR 

SB2-V;0B 

SB2-FREE * 
SB2-PRAC 
^BS-RCAD 
SB2-CM9-r ' 
SD2-AQ , 
SR^-CO.NV 

SB2-srnK 
SB2-arnft 



.09 
.05 

.75 
.02 

.02 

.22 

.02 ^ 

.pa. 

.0^ . 
.00 \ 

^01^ 



.05 

.0^ 

.AO 

" .02 

^ ' .01 
« .17, 
>.05' 

.00 
. .01 
.00 
.03 
*%00 



4S 



23 Observer Conmencs 



'7- 




fX^ General C^a-nent* / - 

2 Student Response ia Spanish 
. 3 TelacHer Action Non-ver^bal 
A 'Backwards Buildup/ Exejrcise 
5: TeacherjRends . 
6/ 'Teacher^^' Respoqaq' In Spanish / uo-n^ai-i 
7 Comment on Back .'of' S.Ummary ^l/eet C7-Cn:HR 



C1-,GENI2C 
,C2^SRSPN 
•C3-TA/nV 
CA-BtjDEX 
C5-REA0O 

c6-tAspn 



v 




. Mean norober of occurrences per; episode. In most instance^ these may be' interpreted as proport lor/ of epJLs</des in whieh the 
event was observed. They will only^id/fo 1.00 when groujjs behaviors are mutually exfclusive ind exhaustive-. 

. • ^ ' ^ . < • • . - ' / 

Binary items are diesignated appropriately as having two caitegorJLes. but only the "behavior 'prescnti)^ category Is labeled 
>. and scored for analysis: - • . ' o / 

_Sone_of the ^Inary Items which' couia be repeated in Vapid auccesgion were simply couo|CAd inste/d of making a separate' ^ata^ 
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A 



^^^n the day-scliool or night-scfiool cla^ses^ Howeve^/^hese will' to 
l^OO^only when the behaviors within a group'^are mutually exclusive and 



exhaustive* 



Note that l:he means fir day- and night-school classes in most 'instances 



e highly similar • ,The categories where this is not the case are: night- 

likelf t 



hool teachers are more 



to work without materials (d.tem 2); day-school 
achers ajjfe more likely to us^ corrective feedback (itBm 12), prompts (item:^ ISTT^ 
aid th^ir actions are morB likely to be nonverbal (item 23); night-school ^ 

tudents^ initial .behavior is more likely to be answering or questioping"" (item 9), 
while"' these behaviors are more likely to be successive- behavior for da^-school 
^ students, (item 20). - * ^ " 

erall, the similarity betweea means for •the remaining categories indicates 

that similar' amounts and the same kiiicjs of behavior .were observable in both the • 

' \ - ' - ^ > ' 

v^day- and nS^ht-school classes. A^description of ho^the observ^tionvdata* 

ere analyzed, follows* . ' ^ * 



^ALYSTS OF THE OBSERVATIONS OF TEACHING^BEHAVIOR 



-A Th^ gystTem of receding classroom behaviors e^ttployed* in this study yielded 
. t23-a^em response record* for each, episode dSfc Interaction observfed in the , 



classrooms' Eaeh such episode could b6 initiated either by , the tealcher or py a 

, ' \ *' I ^ i i ^ ^ * [ ' 

student,. an9< could continue through an extended sequence of dyadic interchange^. 

^ " / \ ^ . ^ ' ' 4 

The participation pf other students in the basic pattern of teacher-stuqent 

' \ c: \ ' • 

interactions was also recorded* Every interchange in' each sequence of 

interaction (episode) was initially recorded and coded individuall^i^hese^ 



in 
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Individual iftgtances wei;e then ag^^^gated by summing to obtain* one datfe-record - 
per episode* An et)isode^ began at the initiation of any new interaction either 
by the teacher or by a student. ^ , i • " 

Methods of Organizing Qyadic Chains of Interaction 

^ As seen^in Table 3.2, some of the thi^ngs a teacher could do to inltxLate % 
an inteifaction with a particular student, group of students, or the' class' were: 
ask a question, model correct usage, or give directions (use qf the Observation 
texicon in Appendix C in = conjunction with Table 3.2 will be helpful throughout 
this section). A student could ln^.tiate ihteraction by asking a question, 

for example, or could continue an. episode of interaction initiated by t)ie : < * 

\ . . . . \' 

teacher by .answering a, question, following instructions (e.g., to read or ^to ' ~1 
« 

write on the board), practicing^ and so forth. If the sequence of ilnteraction 

continued, it could le^d t^, further teacher bel^ior in tlje form. of corrective 

' '» * 
feedback, modeling, prompting, questioning; which could lead; in turn, to ^the 

student's second attempt to answer correctly, to more practice, arid so on. Xong^ 
chains of cyclical, dyadic interactiqn could thus be, and indeed were, coded 
as "successive" b^haviors^in each episode of classroom- interaction*. 
' , purposes of data analysis we chose to" tre^t each distinct interaction 

seqJence or classroom episode as -the^asic unit. This means that all teacher 
^^p^^^^^^^^^^^^h^^^i^^^^^ interaction sequence 
were' aggregated. We thus retained only the^disjt^inction* b^,^een initiating and 
subsequent behavi6rs. * By maintaining the distinction between^ irii-^^ting " 
behaviors and subsequent consequences, however, we left some^oppot%OTd$y--for <i 
the most basic aspects of sequential* oi^gani^ation among Classroom* behayip|:^^ 
to emerge in our analysers. * ? 




* , . ■ , r'- ?-3'7 ' . ' . ' . 

• T« ^^^J'r ■ . ' ■ ■ . 

c . ''"-r^ *. -(Objectives of the Analysis of Classro6m Interaqtion 

W ' 

Onx main interest ia the analysis of the behavioral observation flata 
was to find, if they occur, fairly stable and distinct patterns of^dlassroom 
.in-teraction. Thatols, we wanted to find out if certain teather behaviors give 
rise to or are associated with cei;tain specific "student beha\kiors. From the/ 

point of view of>analyzing categprical datd w^ wfere interested in\)the patterns 

» • \ * . » V* • ^ ' ' " 

of as8o*ciation sh(5j;na between all pairs- of tnulticategory items (subsectidns) ' 

^* given i^ Table 3* 2. " ► ^ 

' Methods of AnaLyz-ing Classr'oom Interaction ' 

■ 1 ' . . • , ' . 1- • 

The availability of data on many individual behavioral episodfes made it 

possible to conduct factor analyses, io*f a large number of binary variables^ ' 

separately for the day-school and night-school samples, even though only six 

teachers were studied in each sample. Sevferal different met-^ods were used to' 

determine how many common factors should be extracted ffom each correlation 
f " • • " • ' * ' 

■•a ' \ ^* s ' 

matrix. ^ r ' ' ^ ^ , , * 

Factor analysis is a ia^ethod of studying the association « among variables. 
* The method uses the inter cor relations among all the original var^-abtes. ' In ^ 

thi& study the ariginal data are « the individlial'^cbdes; that fs,'the entries 
^-.^in .eaeK •tfSf^go^^^ In Figure 3^1. Successive bebavlbrfe were summed x^thiif^e^h. 

episode of interaction: different categories v^ithin episodes are cori?el^ted . 
. tb varying degrees. ~'\ - ." • - ' " * ' 1« ' 

Suppose, that modeling and practlo^ wefe a^oci^ed,^ that is, teachers 
who modeled also elicited practice in the same episode, ^and teachers who / 
psed modelling- infrequently also elicited practice infrequently in the same 
^episode. This asaocia.tipn.JWQuld be represented by a correlation coefficient 



• : . ^ .such a*s £ « .80> "The correlation table (matrix) Is made up ^ of ^numbers, of 

n this kind. JThe numbers vary in size, and l^iere will' be a numbeT for 

bate^ries. Smaller numbers (closer to .00) mean tlie association 
~ V** ^ . • . ' ^ ^ ' ^ 

..is we^ak; tn&<Larger thet nunfbers (closer" to + l.OO to' - 1.00), ^the stronger 

the association. * • 

^ • Fafctor analysis , tries to find patterns of adfiociatio^i in the ^correlajtioi 



^matrix. A pa%terrL^und, for exaftple, ^after J:he teacher asks a question'^and 
i-^^w^v.^;. .^Ij? s.tUdeht attempts an answer., was:"* corrective feedback, prompt, student 
^vffii^^ answers again. This, pattern was found because the three pairs of categories ^ 



were highly associated, .and this association would be. reprpented in the 

correlati(?n matrix by substantial r^s; for example: 

\ corrective feedback-prompts: r = .57 

- "prompts-s-tudent answers? r = .70 ^^i***^* 
\ , corrective feedback-student an^wer^: r = .62 



. .\ Corrective feedback also correlates with teacher -modeling ?r50)^ V 

, o \ ^ ^ " ' ~, ' ^ 

and wileh jstudent^pract^ice (r = .47)1 Modeling also correlates with student 
^ ]|r^aet4-ce ^ \ was found: " corrective feedback, model, / 

practice. ' ' , , ' * ' ' * ^ * 

Thus^ six of the original variables are in tercor related; biit tTie factor 
. analysis breaks these intercorrelations into two patterns or, factors* 
\/ (Note tha>, corrective feedback appears in both, patterns* ) ^ 

The method Is a Quantitative way of iooking, for these patterns » which 

•'1 . . - . . , ^ . • ^ • • • _ 

- are called factors. The methods may yield none or many factolrs. !rhe numberw^^ 
» ^* of factor^ produced depends oA the niimber of distinct jpattefns of association 
in the d'ata^, ^ 



^ ^ * ^ • Factors of Classroom int€(raction * * ^" ^ ^ 



Rather .than discussing factors fouud in our analyses in a rigid but t 



» superxiiii^" way with respect to th^ relative amount of vawTance acQounfed 
for, ^ size of loadings, etc^^, we pre£??"T::tf^^ty to 'convey to the reader the^ 

; ^ • < r ♦ * . * , 

outcome of 'our own 'attempt to infer how classroom behavior seems to be 
organised, idter all, the point of .undetfia^cing these analyses is not. to 
generate numbers that we are then compelled to. take seriously, but to help**' 
jis form a conceptual framework based upon empi^cal results throuj^h which'', 
we can come to understand and discuss clafesrooifiTirit'eractibn. ' 

Comparison of rtaj^School and Night School in T , , * 

^ Patterns of Initiating Classroom Interaction ( ^ , 

Although there are majiy similarities between the" patterns of classroom 



f inter^ction^dentified^bx it Is 

_ .equally informative to note somfe^of the major differences, . It seems that* a. 

major difference .between these samples (or analyses) is related to the- seqaerfcing 

* ' ^ ^ ^* * ' 

^ . / bf classroom behavior. * ' ' . ^ . -{ - ' 

, Teacher model—student practice factors . We found *^tea\:her mbdel"- 



N . . . ^, - .... 

V-"^' rr' ^^^sEuaeht practice^vf actors 'in both samples. In the day-school Analysis ' 



sftDarate "model-practice" fa'ttors break out for the initiation tv)-and 
fpllow-uj), (II) phases of interaction. But in the hight-Schodl analysl^s 
one f actor i( I ) /invo Ived ^o t h phases of Interatftion, while the'other^(V) is ^ 
for follow-up . The distinction between phases of; the "'mgdel-practice" ^ 
interaction pattern in the dayrschooi analysis iSi further accentuated by 



^The Roman numerals in parentheses refer to the factor columns in the ^ 

respective tables. Note , that th^ factors are ^ritited in arbitrary order 
obut aire sequence-numbered in^ tetras of their relative, sizes. ; , 



:e^" -r- 26u. '• / 



/ 

some indication that initiation of the sequence leads to a request to. repeat 
*'^^v'^'^'^t^^"^^Qp^e rather than to automatic continuation of t^e^'^'model-ptactice" 
sequence < 



V > > 



Teacher direct-student read factor ^ Mke "model-practice", another 
^ , major pattern of classroom interaction which^ seems to display, differential 
seqix^tial orgai^ization in the two staples is "tfeacher direct" - !'studeht' . 
,^,read*" In the night school, 'again we find that ;Lnitiation of this seqtience 
(IX) leads to its continued expression or repetition-^but the'*teacher may 
"ask the student to repeat" in continuation of the ^-eading sequence. This 
fifeult suggest rather* sustained, continued reading se^uen^es^ under th^ 
^teacher^s direction, Vith repetition, as necessary. In the day school, on 
the .other hand, factor (IX) suggests a pattern of ^initial "teacher diredt - " 

- ' ' ' s • ' ' ' 7 

1 student read" behavior, but if anything, a low tendency for further X^acher 

^ dixfection or explanation; that is, we see a short ifeading episode which is not 
continued. ^ ' . 

^ TeacheV direct-student read or ask question factor . In^ addition to what 

was said immediately above about the "<ii^®ct-read" sequence in elich sample, 

• • • . V \ ' ' ' — ^ 

_ — ^— " there: seCTis-iro: be another^ sequence (VH dayj-*VIT-Tii%ht)- of- "teacher -direct""*^ 

"student read and/ or ask question." This behdvior pattern seems to' be in 

contrast ta on*e of "teagher ask question" "student answer." / There is the' 

^' further complication of a claas vs. individual distinction in the night school. 

. There "teacher ask question" applies to the class-as-a-whole, whereas individul(l 

students are "directed to' read." One way of interpreting wliat is -sgoing on here^ 

^ is ^ regard this factor ^*s a contrast between two, situations which can * 
^- • • • ^ ' ■ ^ ^ . . ► 

alternate within -the^. same cl^ssrom during any given s^Msipn of, observation. 



In tfie^iight-schodl sample there is some indication^Tiat students may 
be reading mimeo materials but ire being asked to answer questions about 

''i> i ^ \-' ^ V \ ' ' : : ^ <^ ' > ^ % ' ' ' ' , 

, pictures. In . the day school there is 3ome- indicatipn_that tjie teacher^' s 
* ' - . " • . . . 

behavibr:;f^after a student's attempt to ^read pr after asking a question is an 

explanation; this, in turn, may be followed by more reading and qg^estion- 

asking on the part o'f the student—a tutorial arrangement, • * 

Teacher question-student answer factor . In each sample tliere is also 

another, distinct, "teacher question" — "student answer" factor (III day, 

* IV night) that is* not- contrasted with stjident^gj^estion-asking, just as we - 

already saw distinct^Mirect^pad" factors in each sample. In both samples 



it is clear that this soi^ewhat mojre'pure "question-answer" paradigm, is 
^directed at individual students rather than to the Jclass at large. In 
both samples there is some indication that Jthe instructor might subsequently 
repeat the question. , - ' 

Let us 'now briefly name and codify the four factors introduced above 
bfefore moving on to some of the major dimensions of classroom interaction 



whicji seem to follow fchese initiating event's: 1[1) "teacher model^'-"student- 

practice"; (2) "teacher direct"-" student read";. (3) "teacher direct"-. 

* • ' - 

• " . " , — . ' — ' -^"r-r^ """"" *^ 



T ^^tndi'^idual ^uaen^s>ead'"*'BiPask questiori"' vs. "teaqher asks qyestion"-"student 

in class answers"; (4) "teacL'^r questions individual"-*"individual student 

* ; . ^ ■ • • / '-' ■ ■ 5 . 

I answers ^ \ - , «< , / 

t 

Patterns* of Cla^srodm Interactipfi 
^^>- 4jhich are Sustained Once Initiated * - 

— •' ^ ■ ^ /- ■ 

The "model-p.ra^tic^" pattern seems to lead to subsequent* cycling in the 
night school. There ar^ also several other factors* of classroom interaction 



which seem to continue once startec} arid therefore cut acros^ the "i^ts^atlng" 
' ' vs. -U^ubsequent" dichotomy which we have set up.. 



Other factor . One factor which emerged iri both analyses (IV d^y,- 

Il hight) is simply characterized by the "other" category of both initial 

and subsequent behavior' on the, part *of both student' and teacher. The 

observer made a special notation* ^bdut the content of the behavior on the 

^'C^ervation form and the behavior tended to be of long duration. These 

notations were po diverse that any further classification woul<lSiave addeU 

nothing to the analysis. Exajnples at*g^ given in the Lexicon undet "Comments - 

Other." (Stfe Appendix C.)^ ' . A * ^ 

Student-student feedback factor . 'Another factor which sht^wed up in ^ 

^both samples is largely* student initiated. "Student-student feedback" is 

the behavior involved here, both initiating and subsequently continued,, and 

it is associated- with the observer comment that the feedback occurred in 

Spanish (VI d#y, 'VI night). In the night school- tliis feedback might be ■ 

associate(J.-^th "teacher explanation" as well as repeating verbatim what the 

studeiit has eAid. It is possj^le that in this pattern of interaction both 

the .teacher .and other students are^trying to help an individual get something 

Straight^ ^ 

Free response factor . A- factor which, emerged only in the xiay-school^ 

sample^'(VIII) is related* to the- "free response" strategy of ins tmiction as 

opposed to question and answer. .A "freie refeponse"- arises when, the teach^ 

' ' ' ^ • v\ ' . ' * ' > ^ 

has Indicated only the structure and not the idea or vocabulary tO be used. 

Comparison of Day School and Night- School in. / . . ^ , 

,Pa]ttems of Follow-Up Classroom Interaction ' ^ , . * • » 

As for the^ clearly "subsequent" patterns 'of , interaction detected, we 

see two major patterns connected with "corrective feedback."/ / 



Corrective f eedback->model-pract>lce factor * The first factor .mentioiied 




in this discussion was "modelj^^pta^i^e" and v/a stated ^there that it "breaks 
up" into an initiating »and subsequent facfor in the day school, but appears 
as. a cycle of continued interaction in the night school. ^ 

Corrective f eedl^ack-prompt-answer factor * *^other, quite distinct 
pattern of follow-up ^interaction involving corrective feedback was recovered 
in -both samples <I d-ay, III night). -In this case th^re is some ^reason to 
believe that the 'feedback is encouragifig (especially from the day-school data) 
an^,^the sequence gc^s : "corrective feedback"-^* teacher prompt or asked to 
repeat"-"student answer." Since prompting or cuedng fhe student on how-to 
modify his or lier response to make it correct predominates here, we x^ll call 

^ ^ ' ' ! • , , ^ , • 

this factoi;' V*CF-prompt-answer." ~ . • 

However, there are other possible teachfer reactions in addition to .the 

. . ' • J' 

ones mentioned ^bpve; fr.om the night-school data* we see that the teacher might 

* ^ ' ;» ' " ■ ' ' / . • * ' 

- further direct the student; in the day school we .^'ee the possibilitV of qjiestiraing 

the student. . In either case, the- observer comment indicates a strong tendency 
toward a nonverbal teacher actfton, especially in the day school. -Here also we 
see the possibility that other students will get involved in modeliiig, prompting, 
,or 3tudent-sttjlnt' feedback (day school)* In the night school we see some 
indication that thijs particular pattern of interactio"n may occur while the . 
' student is writing\at; the board. ^ * 

. Teacher question-student answer factor . .This description completes our • 

* • . ' ^ r 

discussion of t;he classroom interaction factor patterns except "for one extra 

, "^^bsequeht" behayior found only In the night-school sa^nple: ' "teacher question"- 

^'studanjt answer" jjVIlI). This factor is rel^^tively highly correlated (.26) 

with the "CF-prompt-an3wer". factor just discussed, but "teacher question" is 

' ' * ^ - • » • 

• • • * 

included jon, th^t factor for the day school. ' j 



Classxoom Diff.erences in Interaction Patterns 



Once the factor analyses of behavioral observations were completed', we 

i ' -■ , ■ ■ ■ ■ ■ 

Investigated the differences in classroom behavior patterns — the ultimate 
aim t)eing, of course, to identify interesting', contrasts among -classrooms 
which might account 'for differences in student achievement. We were inte;[:ested 
in between-qlassroom variation in behavior for its own sake, . however, and 

therefore sought 'a technique which would permi4: us to optimally discriminate 

* * ' » ' / 

between classropms -on the basis of behavior observed .therein. 

We settled '"upon the individual classroom as the focus of study at this , 
Stage becTause our interest centered upon teacher differences as well as 
upon differences in the ways 'in. which any given' teacher mighty approach 
students of. various proficiency levels. For purposes of assessing variation 
within clasgrooms, that Is^ the interaction of a particular^ instructor With 
a m[ore-or-less homogeneous group of students, we chose individual sessions o.f 
observation as the basic units of analysis. We contrasted Overall mean 

. " ■ ■ ■ • • . . ■ ' ■ 

differences between classrooms on each pattern of interaction (factor) wi'th 
^the amount of ^y-to-day variktioa^ observed within classrooms for that pattern 

of interaction.. This was 'done simultaneoiusly for scores on all nine factors, 

' ^ . . " - 4 ' ^ '1 . ? 

separately fot the day-school arid- night-school samples. ' 

The technique of analysis of variance was |ised to estimate the statistical 



significance. of the variation in interaction patterns between classrooms to the 

variation xdlthin classrooms.- .This technique yields a ratio of the between- 

classes' variance to, the withih-class variance, which is called the F-ratio. 

'if ' ^ 

The probability of^obtaining an F-*ratio*pf a certain magnitude is provided 



fey standard statistical tables; in Table 3.4 these values have bfeen provided 
fot the signif4.c^nt F-ratios. , _ * , 



■ ^ - 245 



Table present's this information for this study* The patterns of" 

^^f^ interaiction oi^ factors .which we described in the pteceding sections a^*. 

1 listed j^n the left-hand colurtii. ^^^e iF-ratio^ ^r^ lifeted in^the columns to 
«. » ' ' 

* -the fight. The way to read these numbers is as follows: 

,-*=->- . 

1. Note ^that -twcv names^-are- provided in each row. In the^'first row, 

1 ' ' * - . 

tvo different names are listed*, the upper, one being t;he name of 

th"e factor found iri the, day-school sampl^, the lower *being the 



ERIC 



name-'of the factor f dun*' in the. night-school sample.^ Recall that 
similar aAd sometimes identical' factdrs were found in both samples 
In the .neft seli^eral rows, identical names are listed because these 
factors vere found in both- samples ♦ * • ; . 

2. Read across to the columns ikbe]J,ed, "F-Ratio/*^ Reading _in the* 
first row, we^see that the F-ratio-*for "ModelrPtractice" for Hhe 
day-school sample is 4.05;- jfhe asterisk's iiTdfcate that the 
improbability of obtaining a "number this large by charfce^a^Lone is 
' le^s than one in a .hundred (p < .01); - / \^ ' . 

An F-ratio will %e greater than 1 whenever^ the numerator is 
; lai^ger th^n the denbmjnator*^ The nipnerato'r in,^this' analysis is ^ 
the between cinf's'es vai'i^ce^" an estimate of how much* the teachers^ 
'differ from pach .other. "The denominatoi^^^ ^ he^wi thin-classes , 

variance, an estimate of how much each- teacher differs from 
' day-to-day. Thus when the F-ratio is greater than 1, we know 
that the variance .between-cJLasses is greatei^han the within- 
classes variAiict:; or the teachers-.dif f er more from each other 
' , ^ \ than they do from them^lves from day-to-day. 



Univariate Analysis of Variance .Tests*, for 
Classroom Differences on Factor Scores 



Tactor Label 



"Model-Practice" / ^ ' . - 

' • "Model-Prac 1 1 c e-Mode l^Pr ac t Icfe' ' 

"DirectrRead" ' - 
"Di'r^ct-Read" , 

^A^-Individual" vs. "Direct-Read/AQ" 
"QA-ind±vidual" vs.' "Direct-Read/AQ" 

■;'QA-Individual" 
"QA-Individual" 

"StudentlFeedback" ' \ 
"Studedt^Feedbadk" i} 

"Other" 
."Other" 

"Free Response" - 

"CF-Modei-Practlce" . 
"CF-Model-Practlce" 



" CF-Promp t- Answer " 
"CF-Pr omp tr Answer" 

"Question-Answer" .(successive) 



< 



Between l^lassrooms Degrees of Freedom 
^ Within* classrooms Degrees of Freedom 



Day School 
F-Ratlo 



•y- 



4.05 



** 



4,32 



2.33 



17:40. 



7,70 



1.73 



7.88 



** 



11.50 



** 



11.98 



■A '13 \ 
367 



/ 

Night School" 
- F-Ratio 



.83 
.63 
1.24 
4.81 
2.10 



1.20 - 



i.6o 



4". 84 
3.40^ 



■mm 



*p < -05 
**p < ,01 



QA: Teacher Ouestlon-Stvident Answer 
AC}: Student Asks Question 
CF: Corrective Feedback 
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But we mu3t also consider the^pjpbabilit^ that the: 
F-ralioJs value is a chance phenomeiia, 'Therefore, before^ 



?r^* infer that, the contrast, being studied actually* de'scribes ' 
^what/is happening in these classes, we must consider the * . 
^ .statisti(*al significance of ^the^.^Ei^ratio which ±s represented 
. by the^-value. In the case of the first factor,- wfe may^ 




■ -i - 



conclude that the differences/ found between teachers are 
likely to.ije "true" differences';' that is, 'the differences ' 
we have found in our obs^rvatioins describe what is actually 
happening and is rtot an artifact or error in our observation 

system. , . . • . , 

<^ 

/ Note, however, that the Fr ratio for the night-school 
' ^jctor is less than 1^?^= ♦SS, and, of course is not ^ 

statistically^ significant. There is more variaiicm ^sojn^ddy- 

- ' - ** ^ . ' . 

to-day for each teacher than there is between teachers. ^ 
\ ' N^^e^th^Hn the day-school sample the F-ratio is 
significant jfor all but one factor. In the night-school / 

"sample, vthe F-ratio is-. significant only^for three -fattora'. 

-The day-^-^schobl teachers differ 'more* from each other than 
they do from themselves day-t;p-day. The night-schoo^ 
- teachers differ in ;this way onl5|j in three .respects. ^ 



We- will- not undertake a detailed discussion of the univariate ^alysls of* 
- yaiianc^' results presented in. Table 3.4 until after we have completed discussion 
of the mult4variate'^an?iysis (canc^ical dlscrimii^a^t function analysis)*. The 

.'5 ■. 
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latter will pjcovlde an overall t%st df classroom diffenences which takes 
into account any. dependencies among various classroom interactibn factors. 



_:2Le results in Table 3.4.rfce presented here mainly to gvetify our emphasis 
uppn analysing the .day^'^chgol data in what follows, -and rt) give thfe reader * 

^ a familiar frame of reference prior' to undertalcing discus^on'of the multi- 
variate "analysis. ' " ' • 

From the svyavy results in Table 3.4 it 'can /be seen that many classroom^- 
^15^®^^"'^^^ "^^^ detected in the day-school sample but/ presumb'ly du^ in' 
part tro the smaller number of observation sessions per classroom, ^ew difference 
were detected in the night-schooi^ar^le. ' Thare were f rom J.S'' to 33 observation 
sessions per classroom in the day school compared to from 9 to 14 *such sessions 
per classroom in the night school, * ' ' ' 



Session-to-Session Covariation 



of Classroom Inter action Patterns' • - • - 

— . — — 0 < 

*>eecause of the greater precision of day-school classroom comparisons / wb 

. • - * \ • » 

will focus there in the' following discussion with only 'a brief discussion later 

of the night-schoQl results. In considering the Tinivariate analysis of variance- 
results given in Table 3.4, it mus,t be remembered that the various classroom 
interactioi^ factors are^riot uncorrelate^ therefore,' there is. apt to b| some^ 

overlap or redundancy in the F~test results reported. In. order to get a 

\ ' ' * , 

feeling for this association between classroosa ^l^ite^action factors as they - 
vary from day to day (observation session to observation sessiott>*-we can 
examine the poole'd within-classirooms correlation <matrix, amq^ng the factor scores 
which is presented" in Table 3»5. ; ■ * . ^ '\ / ' * 

V . ' . • . ' ' ^ / i ' '" 



"• ' TABLE 3.5 ■ 

Pooled'Within Day School Classrooms Correlations Among Factor Score Session Means 
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'FREE 
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-.14'' 


- *V21 


^- .25 
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7 .25 ^ 


.00 


.13 ■ 


• 1.00 

f 



Mnemonic lab^r ^ JEnteraction pattern 



QA IND . . /. : . . . "teacher question - individual" - "stvident^ answer" 
CF-P-A . . V . . . ; "corrective feedback" - "teacher prompt" - "sytudent answer" 
CF-M-P . . .\- . . ^. "corrective feedback" - "teacher model!' - "studenV practice" 
MOD PRAC . . .\ . "teacher model" "student practice" ' « 

OTHER ' . . "other" ■ ' . ' ' 

DIR READ ; . ; .- "teacher direct" - "student read"" . • ' 

OA v^ DIR ....... "teacher ask question" - "student answer" vs. '^^acher direct" - "student 

read and /or'*f ski question" i 



SFBK 
FREE 
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^ I \ • j reaa ana/or -^asKs quesciion ' ^ i , / i ^- i 

S .\ .\. . .V^freelresponEe'* \ - ' ! • ^ ' ' ^ ^ \ 

■ ■ ' ■ 1 • ■ - ■ . • • , ■ ■ 
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^^"^^ ^^'^5^^purposes of discussion we. have ordered the irows and colunto^s ,of^' ^ 

•r.'t ^ th^ 'T0^c^3:«<h'w±^Mn--c-la9sroom correlation matrix in Table ,3. 5 so*as to best 

\ > *\ ?eye|l^t|L^^Ki^t^rlng of cla.ssroom interaction patterns; Note that mo^t of ' ^ ^ 

' ^ the positive- intei^orr^lations are neaS^the mainViagonal and that several * 

" »: • . ^ , . o . ' ' ' ' \ 

> *f «~ * ^ < ^ * ' * ' • • V * 

clusters of as§ociial?^*^teractioii patterns have been bloclfed off thfere. 
• Note in Table^. S.^^artend^ncy for the "corrective fefeciback-proinpt-answe^ 

' interaction pattern' to o^ur on those days when the "question-answer" .i^att^^^ 

♦ - . ^ ^* ji-"**^ " • • . ^ ^ 

>■ . * » «» 

is used-with^irtdivi4ual students;, (r = .45).' These two factors thus §o together 
to form a\^"inacro" pattern which links individual epispd^s of classroom interaction. 
A sec6nd clusjE^rTwhich emerges has as its focus the "teacher^model"— ^ 

••^student practice" ' ineferaction.pattern. The inclusion "df "corriecti'^e feedback"- 

. - ^ • • ' ^ • \' ^ ' 

"teacKer modelV-" student practice" as well as "other" in this cluster indicates 

. - ' , ^ ^ ^ • ^ ^ 

z '• * ^ 

day-to-day covariation in^these repetitive,' rehearsal7based sequences of 

interaction. ^ ' . , 

• y . . . \- ' • , 

It is interesting to note thdt the bipolar factor which contrasts teacher 
• questioning with student questW-asking; is relatively .independent" of ^the other , 

interaction. patterns as they 'v^ry /from day to day within classrooms, finally, ^ 
' we see that the more flexible and open" patterns .'of ^lassroom interaction , ' > 
\, „ ("student-student" fe^back,'-* "free resp^se") show/a flight tendency to prevail • - 
oh the same days_,(.13) but .are incolisi^tent with the/"-direct-read"._ C--25)/_ .^^ 
"mtJdel-practice"- (-.21) /"other" (-.25) syndrome. ^. * ' *^ / 

' ' Multivariate* Analysis' of Clasfsroom Differences in Interaction Patterns 



• . . • Movia£>rfbw -to a coiisi deration of ^overall classroom differences averaged. 

across days t)f observation we must consi-dfe^the outcomd of canonical discriminant 
^ ftmction analysis. This analysis is designed .to identify those* linear, combinations 




•Wof observed scores, (in thj.s case/ classroom interaction factor, /scores) which 
- maximally discriminate between groups relative to, within-groiip variation^ i 
far as this study is concerned,; the role of canonical discriminant function ^_ ^, ' ' 
analysis fs to identify *ways frr which classroom interaction patterns "can bQ ; ' 

cojpb.ined to^'Vield stable overall contrasts amdng the classrooms, studied; that ' 
is, to find how the classrooms dj.f fer most clearly from one another on the 
average while displayinlg minimal day-to-day' flue tuatioris* ^ " ^1 - 

The essence of the carionical discriminant function technique is to find " 
the bases on which the classes are most sharply discriminated 'from each other 
on the average, '^^he results of this type of analysis are. portrayed in terms 
of disciSiminant axes. There may be several such axes.' The axes can then be* 

; , - \ ^ ' ' ^ V 

'described numerically in .terms of the original ;f actors^. . . ^ 

In this study, tfhe canonical discriminant function analysis^ generated 
eight discriminant functions, but the first two ^c<:buhted for most of^the . 
variance in the factefr score means. Each .factor could be "located" with ; 
respect, to these axes^ These numbers ^re ..pro>jided in labile^. 6 in Chapter^ ^ ^ * 

.Three. ' ' • " * * ' - . ' - ' 

' An inspection<iof the results .of the canonical discriminant: .function analj^si^ 
among day-school claSsrooms.Jsummarized. in Tdble 3.6 reveals that thet^-are two ^ * 

riaior dimensions of variation .between classrooms and two or three more minor 

* * • ' r w * ' ^ . ^ '.' " ' / , ^ , 

dimensions. . \ ' . li ^' ' 4» . * 

*. 1 One way, to describe the results of these analyses is to portray them \ \ 
visually. Each cUss uill be 'represented iii a s>ace def ined .by the discriminant 
' axes Each class will be represented by a point. «. By pr-ojecting, this point . ^ ^ 
out onto the^axes we ^an obtfliu-an idea,* of th^^ characferistic patterns of , . , - . 
''itfferactibh in that class. ^- ^ , • ' . , . ■ , 



Because the first two canonical discriminant functions account for * , ' 
tbe majority (78 percent) of all di*^rences between day-sfhool classrooms, 
in terms-^of stable i^atterns of teacher-student; interaction, we 'have chosep ' 

J - i • ' ^ - . ' ^ , 

to plot ^he classes with respect to' these , axes .in. Figure 3^2. ' In the figure 
each classroom is identified by a letter-riuiiber cotnb'ination/ Tfte initial " • 

; ' ' . . ^ ... • ' ^ • - . , ' 

letters range from A. through. F and .identify fpe six ' day-school iffs true toi^s. . . 
The nupbfers rang^ from 1 to, 3 and refer to proflcienpy level' op the class 
being 'taught as measured by the Jt)hn -Test. The lower case letters identify^ 
(fi^ferBnt classes any given level where they are. taught by the same teacher.- ^ 
..We have-dravm vectors ijn Figute 3.2 to represent the vay^tThe nine original • 

r ^ " ^ ' ' ' 

factors of momentary classroom trfteraction.,pro-fect , intV th^ discriminant- 4P 
space. Note that the important' features of these^vectors are their directions r 
and'..their relative lengths — their, absolute' lengths are arbitrary and ttave been 
scaled merely for cony^iencfe of plotting. By v£^^j^g the direct perpend^ula 
projection of cla^Sr^m- cencro;Las' onto.^these ^ectors-r<^ through the, .'^^^^^ 

-among 

el^sraon^^ ^.^g^^^g ^^ -r^^^^P^ '^^i^^^^ of %i^^^Pivrj^By^ rela^^- 
collinearity or perp^endidul'irity'o^^ can get /a" feeling |or ^ 

hdw classroom bj^havpLors associate qV disassociajLe ^n the profiles^whicji", 
aistingiiish' cldsa^'ooms from one another. ' %. ^ ■ ' ' * ^ . . . ?v . 

[kd for the classroom cent;roids plot'ted in ^Figure 3.2^ the most otitstanding 

^ '/ ' . ' t - ' ^ A ^ 

featur^ i^ th^ obvious -clta^et^l-ng of classrdoms taught *by the same teacher. , 
This ^ph^pnomenoi^^is partici^rly striking in .view of the fact that no informalion 
'abCtut'who taught what (jlass ^.tered into anyjof the apaiyses If^ing. to these \ 
result's. The evidence, is ' tBusT incontrover/iblte that teachers have consistent- 
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Ploc of Day-Schoql Cla3&room Centroids and Classroppi Interactiojv' Factors 
-in #he Space Refined by ^the TWo Largest Canonicai^^^^i^cro^iant Axes 
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and distinct "styles" of Interaction with students — styles which do n0t in 
general vary markedly even when teaching ^classe^ of quite different initial ^ 
ability level , , , ^» \, 

V .... - • * • . 

Orily one teacher, C, appears to employ widely different strategies wjien 

c > ^: * C * * ' <' ^ . 

teaching students of differing ability levels; however; it must* be pointed 

oiit thkt not other tea#ier had the opportunity to teach classes containing 

» ) f • y • * * ' ^ * 

■studen^:^ pfCsuch widely "different ability levels • ' . 

Because, composite variables in canonical form are notoriously difficult 

and dangeroiis to interpret we will not attempt to label the axes in Figure 3.2. 

' \ - . ' 

* ' » 

We can note, however, that the first (horizontal) axis is a contrast between 
about the ,sam& clusters of variables that we identified in the pooled within.- 
classrooms correldti()n matrix in Table 3.5, That is,, a general pattern of 
DIR READ,' CF-M-P , MOD. PRAC ^ and OTHER is contrasted wlfh QA IND and CF--PrA. 
Wet anotlier method v^s used to transfoi^m (rotate)^ the' canonical variates 



'xjtnto-^.-mare tkeojreticaliy informative and interpretable positio^^^ For fhls 
purpose-we chose to use only the first four discriminant axfes since they-account 

for most (93 percent) of the variation between groups, and little significant 

i - 2- • " 1 ' . 

variation remains (x - 59.4; 45 df) after the fourtR axis.is'.cotisidedred. 

(Table 7 in Chapter Three presents the ^DIRECT GEOMI]^ transfprihed between- 

classrpoms canonical-Variate factor pattern matrix. ) Upoi^ transformatfiori of 

the first; fpur betweep^classropms canonical variates Wje came up with ,twp^ 

'major contrasts between tea^hef-st^udeivt * irfter?L«6tion patterns .(still accounting 

for the majority of 'the betwe^n-rgroups variance) and two smaller variates each 

delating essentially .tp. only otti. ot^ two patterns>«f momen^ar^r interaction.. 

Because the transfoi^ed canonidal variates are fairly highly Vorrelated it is ^ 

clear that-^here is a. second order ^ganeral factor in operation here.. 




It must be kep^g^ mind that the four contrasts preswted were arrived 
at because they discrimnate quite sharply between behavior patterns observed 
in dif f erent/ day-school classrooms in this particular study. We have already 

7 - V • , • ■ • , " 

seen t^iat the classroom diff erences« detected are largely a function of teacher 
differences, however, so it is quite likely that other ^contra^ts in teaching 

behavior would become salient given a different sample of teachers.? - > ^ ' . 

•i ' ' • ' ' _ 

' First axis * The-first axis is a specif^d distinction between the "question 

answer-corrective feedback-prompt-answer" paradigm of classroom interaction and 

the "direct-read" and/or "ot^her", paradign^,. At the ievel ^of contrasting iiitlividu 

aspects of classroop interaction; however, interchangelbiiity characterizes the 
* * ' « 

' follow-up aspect of the positive pole of this cqntrast.' A "quest;ion-an^wer" 

pattern directed to individual students and followed by, ^corrective feedback" 

' \ /\ J ' ^ • ■ . , ' * 

may lead "to quite »a variety" pf 'teacher behaviors in addition to "prompt", 

^ . • r' ^ * ' \ , 

^i3tudent-student feedback"' likewise .figures prominently as a lollow-up rette'rn, 

as does ||||udent: ^ues&lon-apking". The "direct-read", "other" pole ®f rhis 
* f i;:st contrast ^^etween clasjsroom interaction' patens is aisp^haracterized* 

by many* subtleties, at the ievel p£ individual ep^J^des of interaction, 
* Second axis . The sec Jn^/* transformed discriminant '^function axis picks^ up 
" a contract between the "free response" mode of interaction and^' the* "model- ^ 

practice-ccwrrective f'eedbeck-raodel-praotice" par^digm.^ As f9r aXl^four trans- 

♦ . V ■* ' ' ' ' . . ^ 

' formed a^es*^ th^e positive p/olq applies more to iii'dividualized instruction* ' 

While the. negative 'pole refers- C6 class or group-ijriente^d instruction, 
'/ • .Third) axi s. ^The third transfonned cariohical variat^ is, npt a *stron-g . ; 
- 'bipolar -contrast, unlike the other^^ but -relates quite simfjly^ to th^e presence 
of "student-student feedback'' as a component of dLts^rD'Om interaction, i Along 
vwidrchis stuaeixit feedback. i& bome' indication of. corrective feedback on^ the' 
. part of the teacher ai\d other foiluWp ieh^-C^i^rS similar tp. those seen ^dx" ^ 
^ "the first axis, ^ . , C io 
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. -Fourth/axis . Finally, the fourth transformed oanoxiicalVvjkriate is 

interesting in. that it relates strongly arii negatively t;o the/ original 

' / ^ ^ . • ^ NvJ— ^ 

Jjipolar contrast found when" we factor analyzed classroom in teraction'^p^ terns 

■ r y ■ - - . . ( / ■ [■ 

as they varied from episode to episode: "teacher question"-"student answer" 
^s. "^acher direct"-studqnt 'read*and/or ask question"! We now reit^erate o^r 
earlier conjecture that the ^ direct-read/ask question" pole cTf this' fact;or is, 
indeed , "an interaction ' pattern characteristic of relatively, gr^ja , yafrtructuted 
Classrooms,, whereas the " odestion-answer" ,pole characterizes more highly . 
* &t;ruc^tured classrooms^ 

JJhat the. traiybfdmed canonical discrimina^nt function axes seem to repeal 
,.is .that several patterns manifest cMssroom interaction 'are combined in 
different ways to forj^ "higher-order" patterns. Thus, fche observed interaction 
pattern^ "teacher ditect"-^*st:udent >read" can either be^ pa^ of a highly structured, 
* prograanned" classrooij interaction climate (the negative pole of the first axis) 
•or part bf a'mdre open-ended-, unstr.uctured ^^ting for interaction (the^ positive 
pole of the fourth axis)*. Looking at the opposing poles of these same axes, ' 
jespectivel^jvre can irif^r thlt "teacher ask-s questiup" can likewise be part 
of an er^cfour^ging, po^mpting, follow-up pattern or J.t can form part of a more- 
demanding pattern' of drill. • ' ' S " , ; 



' I . '^^tpmar^^f the Results. of .£he** Analysis of 

, ^ , • Lha; Observation of' Classroom Interaction Patterns 



. At thi,§/p^int it will be useful to silSmarize the major steps in the analysis. 



' 'The f^ftrst step prodji'c^d uine patterns of interaction (fai>tors). Since^^nthese 

T-r ; . .- \ .■ ■ ■ ^ ■ 

nine factors wer?^,intercorrela.t^d, h second analysis was conducted, a canonical 
/ ^ dJ^sCrlpiViant^ f^incciori <^k\<xl j .^ , T1i<^ aiae factors' could be JLocated principally *• 



in terjns ,of two discriminant analyses. Each class could be described in 

terms of its 'location with respect to. these axes and with respeo^t to the 

.ariginal nine factors (see. Figure 3.2). These canonical variates were 

ther\ transformed with respect^to four axes, two of which* account for most 

of , the Variance among the canonical -variates. ^ 

TJie four axes produced in' this^asc analysis ar|* b.ipolar; which means ' 

the data .for a sec of observations will reveal a position closer to one end 

f 

than the other, of an axis.. Thus by knowing a class' or teacher's discriminant 
''function scpre's Vith respect to these axes, we visualize the cla^sroc3m inter^ 

actibn patjtems in that ^ass. ^ .* * 

The first discriminant axis 'contrasts two paradigms ; 'one, >, the "question-" 
' answer- corrective f eedb/ck-prompt-answer" style; the other, 'the "-^irect-read' 

or other paradigm', /he first paradigm is a 'structured style, almost as if 



it were progr^smed. 



The other paradiem is les's structured, requiring more 
responsiveness on tHe pare of the -studeats.^ • . • 

The paradigms are* not "pure" in practice. The follow-up phas^ of tha-^^ 
' first paradigm lias several variations. Other padtems occur as part of it- 
jbsuch "student questions", ."'similarly^ the "ALrect-read or' other" style 
has- components ,of Tteacher explanation" or '.'ifodeling" or "writing; on the ; 



I. 



. bodrd 



d" In it.- principal features of ekch ,^ole Qf tKis axis are> those , 

listed in its label. • r ^ y - • • ; . ' 

" -The second axis contrasts -a "free response" mode jd-th a "model-practice- 

\ Wrec/ive |>»«tlljack-mo del-practice" para^i^ the m6st g^timctured of 'the 

iAtera'ction patterns. ' Thus what this as well as the first, axis seems to' be . 
' .describirig is" a.>c9ntrast between a more' dnU a- less' structured- s tyle A ^ 



' ''(9 ■ ' - . ' ... * . , ' \^ ■ ^ ^ ' ' 

.Since classrdom patteriis are not pure types in practice, -a dass bry 

a teacher will usually havej scores that place them 'closer to a pot^^JL^ach 
axis. Teacher A andji^ache^^ B will be alike o?^if fereat;^depehding on 
Wher^ th^y'are lod/tid with- 'respect -to these axes. Also any one teacher . 
may include othel^ patferns within the overall atyle. ^ - * . 

i * Th6 ttfii^d axis is'not a strong 'bipolar factor. ;The axis locates a 
class with respect tp wheth^z^f'^tl^ feecfback" pattern appears 

in the interactions. \ 

The fourth axis is a contrast between "teacher-question and student- 
• answer" and "teacher direct-student read or ask question." The "c^uestion- ^ 
answer" poM is characteristic of the more structured classrooms. 

* It may appear to the reader that the classroom^intferaction patterns 
^ simply are eitker structured -or unstructun^d.^ Such is n6t literally the 

i case. There axe' three paradigms of structured instruction and two of 

■I ' - . - ' ' ''^ " ^. - \i ' 

' unstructured instruction. But th^^e paradigms obviously do ndt [occur in; • 

^ • • ^ * ^ Vi, ■ ' ^ ' ^ 

/ purel form in this: skmp]^ of . .teachers, and) seujren|:s.. , , - , | 

^ . \perhaps the most illumina'ting. wa^^t^^-^^^^ini^^ these axe^ mean" Is 

■ to think, of' them as dimensions along which a- teacher* will have a score. . Thus- 

■ each teacher will, have a profile of scores. 16 is ^this profile ithat describes 

', « • ■ ^ ' • ■ .■ • ' ^ i ■ 

' each teacher's criaracte#Lstd.c style. 

> • ( 



> Wha 



t is most! important- to notice abo'ut tthes^e sale's ji's ' thei^ stability. * 
\we found -that differ ences^b^^^^n tiachers .wer^greater than the tebcl^rs' 

- • ■ { '■' 'J : ■ • ; - - '^-^ V ' . • -1- 

- day-to-day variations. Furtlfer, with one exception, the^teacyi^r s style,-- 
• did not vary with inspect to the proficienqy level of thfe classK • 
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. ' * We^now.'turn to whether these styles, had differential effects "on the 
acqui^Ttlbn pf " prof iciency in .English .l^y the students/ ^ First) however, we^ 
will-describe the' reliability of the. observations • Had they not been. reU^able, 



we wouldf Wst likely ndt have found significant relations Vetween ciassi;ooTn ' . 
i|xtei>actiop patterns if they existed. * 



fReliabiiity oJi» Classroom 




action Patte^rn Coritr.asts .v ^j. 



re.'s^cession of niata reductioa.pxp.ca^r;es *^ppiied* tq^ classroom^ behavioral 
observations has ultimately 'l^d tfd oply 56 et^entX^l ,nu$fcers > ^the classroom 
centroid coordinates on^^feacrf Jour tVansformgd ' dj-scrimii/aoK function -axes . 



^The^e coordinates^ define four ^contrasts^ 

e^ict sfei 



^ayfschpo^r c/a^rooms which 
can now be used in . an attempt to predict sfeudeftt ^ach^ev^S^^ th^ school ' 
\^,,jy^ar irom classroom^ int^ractfoiCfex^e^iencesl .Hp^^ti^h^^ 

' ' I of classicoo^ .interaction' sin^^^ The 



issue 



of irferiabilitv' alspfgro^xi^^s j^^ to fSrturn, as premised, 

I t'ol.the toivarla^T^anaiysis .a£^^ since 
L wel win! discuss re^abili^ ijft [t^rm^ . ^ 



^ J , Components of ^Qrtance AnalQrsi^'. 
7^' ' of Classroom *Interacri!on Factors - - 



. - r Theld^y^=^5t^ comprises- a. completely Jieated-fo^^^^^ 

•"/J "aCyriie design. tkyiivg%^r-ciimateiy' 2^^^^ each sJ^sion.- 

A ! WToxlriaU-ly 25 sessions per -cla^sroom,/and aBoutj' two -clas^ofems lie'r teacher. 

: c'r -ifi - » • - V-"*" «^ . -.. .-C* 

JTne^eStilts or^he analysis, of -variance are prejifentR'in^^blfi, 3.8. The ^ 
-"wciginil ftin^.^actors derived £rqni"t;he first' f^ct^r aria"ftrsis; #i*e listed at the 




.-Wi^^i^'^eaclJ colunin;. ' Down-, .tlie- left-hand Iside atfe the ,source^, o^f variance. . The 




^* ^ ..5 • - « ■ - • ^ .... / 



TABLE. i.'S 



Componencs of Unit Total Sample' "Variaace of DaV-Sc^iooi^. Classroom 
laceraccion Faccors Due to Teacher, Classroom, Session*/ Episode Differences 



QA rjli CF--P-A OTHER -^DIR READ 



Source . 



Teactier 



ClasS'room 



16.0' 



J 0-0 



Session.*, * 
Episode 



9.4' 1. 



.2 .., ...aj]i- 



0.0 



< . 15.2 12.7 30.1 24 .,4 
^ 68.8 77. (r 68.a 



i 



69.7 



FREE -MOD PRAC CF-M-? 



11.6 ' 
l.&^ ■ 



4.6- 



OvO 



-; 

33.9 11.0 



Discrininanc , +1 • +1'., 

** Axis .. 



-I 



52. 84.. 4 

^^^^^ 

'-I ■ .- ( * Vii "- . -li" 



4.6 



■ 1.5 



10.8 



83.1 

-11^ 



SFBK 



V8 0 



7.5 



90.2 



+III 



QA.VS DIR 



/ 

1.6 .' 



33.8 



64.7 . 



•rv 



episode- is the smallest* unit^^aT-analysis ; episodes ,occuf within Sessions 
Recall that an ^iso-de is la^i^stlnct paftern'of , interaction. Sessions^ 
are the hours of observation om different days, .^ejassrooms are the different 
' gf oupf taught a teagher; and teacher, of cdurse, refers to the indi3*idual'- 

teacher. • ^ , •'.i— 

The numbers in the col«ums are' lpercentages — percentages of the variance 



in the factor 



Dutable to* a sm^rce of variance. €onsider^the factor. 



"Question-An&wer--Indivi^ar' 'CQA^ IND) ^ «^ This .£actor comprises, those perf o.rmances 1 
in which-^tbe teacher ask^ an incl^vidiial student a que^tifop; Teachers vsffy^n 

how much th^ use this'performances Some o^ this variation m<ay be as^so^lated - 

? - ^^y^^^ ----- ^ ^ 

with episode^; thatvis, ^^pme ep%|pdes will include- this perf ortnaftc^ while others 
'will not. l^ether 01^ not tAe performance ^pears-may be due td^what the teacher 
/ is trvine to d(> at (that mofmfent. ^Or the variation in th^ data may be due- to 
{ 'the day or time of the-^period ih which , the .class^as 6bserved. Qr, the vatiattoa j^^'^^^ 

!may be' related to. the xlass 'being feaugtrL- ^Or, it ,may beNrelate d ,to dl lf^enggsT^t y ~ 



fin teacheirs* styles'. 



V.' 




'By reading down the column unAer QA IND we can see what^percent^ag^/oT ^He"^ r— r—^ 
* varianc4 is attributable^o each s,p«/ce. ^Thus epl^od^st account fot 69 p'ercfent""**^ 



i 9f -tiie variance, 15 percent is accoujited -for.b.y session , and 16 ppcent^' is 
. 1 attributable to .'teachers. But I'the 69 percent attributable 'tp episodes is not , 



/ 

< 



) Statistically -slgnififcant. Hh^refore, the variation in this peI^f ormanc.e, is 
related to who- is' being observe^' (teacher) and when they ar^ being observed^ 
(ses^ion^, Th^ teachers dif/er^ from each other in this respect; and ^n 

^ * /' • 'tc ^ * ^ 

^ individual teather will differ in this respect* from ^day-to-day. 



■ V 
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From'Table 3.8 it can be seen that from 53 to ,90 percent of the total 

m 



varianceof each cla'ssroom interactiorrf^p^e^^u^^ at tribl^ed* to 'within- — 

» % • , ' * w ' ■ ^ — 

' session variation (see line lal^el&d ,£p:j^ode in '^ab^3-8);, that^^fcS;.^^ 
variation ^rom interaction episode to^, interaction epl^>stde< during the s^te^. 
session (4ay) o'f observdaon. The. fact<>rs^ ate . thus^ ^fr^ factorg of 



withiil-session variatioit; 



{J 



In Ta:l^le 3. 8. we have " indicated the* protjability lavels* ^ssocia^fSd with 

each component of the total sample variance, along' witffr the. associat*^ degrees- 

of freedom-. From these results mssuay aspects of 'variation 'in classroom 

•If ^ < ' . . , 

interaction, patterns pan be cUyif^d, It ^is. pie^af", /or* i/fs^anc^e.. thatz only 



4 

^ very,si|aal]fknd' ^en^rally nonsigndf iVant cont Abatipn \o Ivariation in .'^Jrtea?ir 
action pafctdrns can be attributed t\ different iai treatm^kt* of 'karate 



cla3srobms "by xhe same teacher (see line labeled! Classroom in Table 3,8) — an 



^rJ^^9^® already suggested "by the compact clusts^ring of classroom centroids- 
: SiFimCst. -teac^ in Flgjar^^ 3.'2 of th^. mult^-variate. analysis. ' • ^ 



_1 — Thi^ relatively . la;c:g,e . components gf. variance' for. Sessions suggest .thit^a 
. iJ^Haber^af ^yjgse_f;actors_fitictutate s^ibstantially^frjDm day-to-day. Especially* 
_ tja^warthy^.ake. the laj^gipis^mpon^^ v^rianc^ for- interaction- patterns" ^ 

^^"^V"di*;|<^ibn**'^24".4'p^r^^^^ 33.8 percent) behaviors., The^se results suggest 



that, any given teacher might be inclined to d^ote certain' sessions to these . • ^-^^ 

^ "specialised", activit'ies but i^-efr^in from doing so in other sessWs, ' . ^ • 

^ But, cons^^erat'ble session^ tq^session. variance characterizes ^many of the j * 
classroom int;eracti^ factors, this result indicates that each session of 

' S^ — -- - ^ •^-^•"iv / - - - ' _ ; . ' ^ ^ * 

-% , Spm^ of j;iie sessipn~t^-seasion .flucfcuaLion could be attributed to obse'weeJT* 
, biases^f since varioVs .obs^^ers .ffoLIect^^^ diff^yknt se^ssj^^ This* 

^^k^ hov^ver, Vheri; we cdn-^idfet't^^^^ \ 

. ^'-o^sej^e^ appfoalh ItD* reii^^^ < ' -r- ) - ^-^ T,? i7 * 



'obs'ervltion provides a rather narrow Wiew of what is going on 4n the classroom, 
» , ' **** ' » ' . * ' ' ^ * 

in. general* and may explain wfiy ste^ble teacher differences could not be veil 

V ' . * X • / ' ^ 

^established'with the limited nig-ht-^chool observation schedule-.* 

* , Ih stnranary, differences amgng. teachers account- fpr .fxom-^,5 to^6 percent ^. 
of the^ t Ota 1% variance ip.t>he observatiollal^ dat^ (see linjB labeled^ Teacher ^ 

in Table 3.8); '-lie differences among 'teachers are statistically .significant 

' ' • -'^^ ■' '"(V. ■ '• ' ■ » ' ". - " 

on alV-*ut two factors ("SFBK ah'd QA V& MR). ' In ordinary language ..the' teachers • 

t^*h differentl-yVand they differ most on thfe'' factors "questi6n-&nswer--icidiv3.dual 
' • ' ' ' * ' » • - '* ' / '• . " . • 

• student"' (QA'IND) and "free response" (FREE). There is>pt much evidence thdt ' ' . 



tbe particular _classro6m ^being taught by a teacher has an^ inf luence, upon .the *• 

^ -.L,^ . j—^.- ' ^ ' ft" " V ' " --^ ^ ■ ' ' ' ' ' - " 

. ..patterns- of iriterapt-ion which tkke>lace therein,' as. we hqve seen ^ earlier in ^ ^ . ^ 
J' , «^ - ■ - . • • "' ♦ . • ) — ♦ * ^ . ° <^ • ' ' ' 

connection with FiguCe; 3.2« The majgr component o'f overall, variation in classroom 

- interaction patterns 'can be .attribu|e4^,. alten^t^^on^^in^nteractlon patterns ' • (» 

from episode-to-episode," which is no1^~alftpri-sing,'- but sessioVto-session changes, 

• aiso'-play an "important role, ' Mot ice that session diffce.ences are >t3tistica;iy ; 

' ^ , ' ■ * ' .' ^ 

•significant on all nine 'factors. • . ' . '* ^ . *■ V 

. ' Fortunately, prediKftg wha't inigtit t;&ke' place-, in Sich indi^i'd;|l- classroom ' , 
" eMsode or even in kny. Men seslign of ' bbser\5atfon was not our task in* this ; ' . 
ktx/dy;"^so^e"can be , conten^f o ^epoaint £6r suc^h variation %y5:ef er^ce t|' ^ ■ . ' : 
'l^ypothetical determinants/or ."•^a'ct'orsi' of 'momentary classroom interaction. ^ < 

• Our 'aim was 'to detect, stable clas&room or-teacheif ^Ifi.^erences ^and it Is ^ • , 
' assess our attainment oS that gO^l that a^reassegsment of the -components o£ 

%^riance in Table 3.8 is.Vequir.ed. . T^t significant teacher di£l:e'rencesvere V 
• obtained despite th^ fact that much session-to-sesSion and episc3de-to-epis,6(i6 
^yariation occurred Is not hard to* understand- because so m^y '^egsions and * 
."episodLes were recorded for each teacher. ^ ' r • * i * ^ . • 
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* When the components of variance in Table 3.8 'are ;i^eighted to take itjto - 

' / ■ ' 

"accoimt the fact that apiiroximately 1000 episodes of ;lnteracticm were observed 

". , . \ / " • '■' ■ '' 

per day-school teacher, about 500 per classroom, ana about 20 per, session, 

we see that a very large fraction ol the observed variation among teacher 

means on each classroom interaction factor Is reliable (i.e., can be attr^rbuted 

to tru6 differences between teachets>r''''''6ince^ubst^^ the saiSe issue 

is addressed by the F-tests among teachers (given in iable 3.4), it is clear 

why so many significant results were found in the day-school sample. 

Components of Variani^e Analysis of ** 

transformed Discriminrant Function Scores - ' , • 

The multivariate analysis of variance (canonical discriminant function • 
r analysis j^^ee^Table 3.6) revealed only a few significant contrasts among, day- 
school classrooms. From 'the first four of these canonical discriminant functions 
we derived the four transformed axes dis'cussed earlier. Sinc^ these are the 
classroom interaction contrasts which we wish to employ to predict student 
achievement, it is in order to determine how sensitive teachefr scores on these 
contrasts are to various possible underlying sources of variation: • teacher 
^ differences, classroom differences, session differences, .episode differences, 
and observei" differences as well'. . The 'issue of inter-observer differences will 
be taken up shortly. ^ . * , 

First, howeyer,^let us look at the results of a second component of 
variance analysis for the transformed discriminant axes, just as was doha for 
the nine original fa'ctors of classroom interaction In Table 3.8, The * upper 
section of Table 3.9 gives percentages df variance for the transformed canonical 

. . ' . ' ■ 

dlscrimitiant function axes comparable to the entries in Table 3.8. From these 
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TABLE- 3.9 V- 



^ompon'ents of Unit vTotal Sat?iple Variance of Transfa^cd Discriminant 
Akes bu^ to Teaqher, Classroom, Session, and Episode Differetices 



I'l 



III 



IV 



Source * 

Teacher 

Classroom 

Session 

Episode 



22.9 



.8 



13.4 
63.0 



25.7 . 



.3 



* 

15^.8 



58.2 



A, 5 



2. 1 



7.1 



86.2 



A.O 



1.0 



79. -9 



A* 



Relative Gontj ibutions of True Score Dif f orcnres Between Tcncliers, 

Classrooms, Sessions, and Episodes to Observed Variance Among 
' Teacher Means on Transiformed Discriminant Axes 



• 1' ■ 



II 



III 



TV 



Source 



p < .05 



/ ** 

p < .01 




97.2% 
* 1.5% 
1.1% 
.2% 



98 . 2% 



, 5%' 



1.1% 



.2% 



79.8% 
16.5% 




•,/;.- • ' . 

:ies we can see that there is substantial. eplsode-to-episode variation 

^ :. ^ . \ , - ' / ' . 

in all f o.ur of .the classroom interaction cojitrasts (from 58 to 86 percent 
of ^the tot;al variance). Note, however, the large components of .v£ri3nce 

attributaoile to teachers (Table 3*.9, upper section). . 

^\ - ^ ^ . 5";?' > ■ ' 

Ifeachei^ differences account for approximately one-fdu^th of .the total 

. ' ■ '-'^ • ' 

, observed variance cfti each of the first two transformed iaxes (I and II) • What 
this thean's, inV practical ^ense, is that we could' reduce our uncertainty- 
aJ)out which poleNpf either contrast might appea^'^in any given episode of ^ 
classroom interaction by a substantial amount merely by knowing which teacher 

is inrharge. If the teacher is high on the first contrast (feacher B in 

' ■ . - *" • 

Figure 3.2), we would do well to predict "qiie&tion-answer-correetrive .feedback- 

, - ' . . , - • . /' /• y . . 

prompt-answer" inte^raction (episodes. Oi^^the other hand^ Jtf ^fte teacher is low 

* * - . . - ' J' '•^ 

• ,/ 

on the first contrast (teacher F or perhai)S E) , then^^^e would do well to 



predict "direct read" and "other"^. epi^od^s; ^ / 



- Considering the second contrast\we .woul^d Jbroceed in a similar fashion, 
predicting the "free response" mode of^^tidfVidual^lzed Instruction for the • 



positive pole "(teachers D, B, and -^)^J afcl/gr6up. "mod^-practice-corrective 



feedback-model-practice" chains ^af^^|^odes ^fpr the negative pole (teacher F). 

Even the last two transf omf4/^^nical varia'tes ("student-student feedback 
and. "direct read and/or ask'4^^^jP|^^^s show 



^ / 



larger teacher ^d classfoom^^Sittiljqperits of variance ^n Table 3.9 than jio the 

original ^factors of :class^oC |?tfera'Gtion from which they are largely derived. ^ 

However/ it is clearr .that the latter are not -major contrast's in teaching styles 
'' ' ^ i > I, ' 

■ ,v - \~ * 0 \ - / 

so we could' not use knowledge of average teacher performance on these variates 

' ' • ' ' ' ' ' ^ ' : ^ ' : '' 

to pr^dicJC , touch about indHLvidiial episodes of cliis'srbom interaction. Let us ^ 

turn, therefcTi:^, to a consideration of how reliably the teacher me'aps on these 

''contrasts can be estimated per se . '. , / 
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The lower set^tiory'of Table 3.9- gives a breakdown of the weighted 
contribution of each. factor to observed variation in teacher me^iis oh each 



7/ 



transformed discriminant function axis. The first two contrast's are almost' 

/ o ^ \ 

perfectly reliable indicators of teacher differences; 97 to 98 percent of 

*./ • » ' ' ^ 

the observed variation in tocher means can be attributed to true differences 
among teachers. The last two contrasts are somewhat sensitive to' differential 
interaction in various classrooms 'taught' by the same teacher^^ 

The components of variance disclussed above reveal that^e are dealing 
with highly " reliable indices of teacher variation; the relialJilities of " 



1 



teacher means on transformed discriminant axes range, from ,80 to *9o in 
Table 3. 9% But a more importnaj: feature of this jLi{ve;?bigation'^is. the evidence 
it provides that those classroom interaction cQnjtj^^ts which. sho^ dif f'ereittial 



tr/eatmeut of dif ferent cla'ssrooms by the same teacher (are d^.agnost 
in\he ability levels of the classes 'involved. >7hereas our/initial° impression 



stfc^of variation 



of Figure ^3. 2 holds true — teachers ^do have distinct. and cbnsistent; styles oi 
interaction with students, ^there is alsd evidence tHat certain more limited 



aspects of classroom interaction vary from classyoom-to-clagsroon)^ taught 
by the same, teacher . That the latter variation m;LgHt be in re^ohse to 
characteristics is of grie^at ii{tfej:est -and will be taken up later* y 



' \ , In/er-Ob^rver Reliability * ' - 

' ■ / ■ • I • ' 

A final question about reliability remains ^ before we cdn proceed 
main task of relating student achievement to teachet: perfprmance: do 
obse^ers recprd the same aspect of variati9n in classroom^ ^nter^ctio 
In- order to study the issxie of possible* obsein/er , bias, da^t^ was col; 




patterns i 



'ctad in som;a 



Q'O 1 



of the early ofeservation sjessions by pairs of . independent observers observing 
the same sessions. Si^ite three observers were used in the study ^ ?t. was • 
arranged to have concurrent observations made by each of the three pairs, in 
eight • different/classrooms, ^ ' . 

We are^ot in a pc^sitiqn to^ make a strict comparison of inter-observer 
agreement in the usual sense, since observers worked independently and there 
is n0 way to know which of their data-sheet entries should correspond.- (The 

ilqvance of this correspondence is ;f requently ignored in estimating inter- 
observer reliabilities.) Ha%yer, We can compare data on the basis of . 
individual sessions. ~ In Table 3.10 we have^summarizei the results of this 
comparison for each of the three pairs of observers, using as the basic unit's 
of analysis, session means on the four •transforra^a^'classroom interaction 
contrasts. It is appropriate to look at possible observer influences on these 
scores 'because we have used these*^cores to predict^ differences in stude'nt 
achievement.^ • . ^ ' 

Note that we are discussing observer assessment of session-to-session 
variation in these interaction patterns,' but we ha,ve already seen from 
Tdble 3.9 that .enough, sessions of observation were obtained in the day-school 
sample to yield highly reliable teacher means despite any sessiori-to-session 
variation. Since each classroom was visited by all observers, ,the influence 
•of any ppssible observer biases on classroom mean scores is indicant ed by ^ ^' 
session-to-session , variation > which a$ we have seen, accounts 'for less of 
the variance than teacher -differences (lower section bf Table 3.9). 
' Prom Table 3.10 It can "be seen that s*ession-to-session correlations ' 
among transformed discriminant function "scores derived from the data^cdllected 



TABU 3\10 ' 



• . • 11e.1n«, Standard Deviati nnd Correlations for 

Pairs of Observers on Transformed Discriminant Function Scores 



Variate V-K 



1 



1 



"2 



Observer Pairs . 
1 .3 



. 2 



( 



_* 
•X 



.93 .70 -, 
2.36\ 1.89 
.98 #•* 



It 



X 




, "'.65 • n5A 
. 1.01 1.13 



.95 



^ .47 .08' 

y'L42 \9i* 



/ 



.66 



13.- .''32 
1.62 1.47 
• .99 



.00 .04 
1.39 1.73 
.96 



.08 . .59 
1.82- 1.82 
.81 



..•33 .52 
1-45 1.5.8 
\ ..99 ' 



• .13 .32 
1.17 • 'LS?".. 
. ' .98 

.26-^ . .47 
1.54' 1.03 
.37 



/• 



• sa 



.87 
: 2.44' 



.55 



.97 



- .13 - .09. 
1.50 i.65 
- .97 ' 



- .11 



.2-3 



l.;83 1.63^^^ 
.97 



'vx V..m.eaui.'''s. = sUdard 'deviation, t = product moment correlation;' 
eight' 'Dl>se.rvatiQiji^v.' .~ • . 



4 ^a" 



V \ /' 

t 
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• L ^ 

concurrently by different obseirvers range from .'95 to^. 99 for all but the' 

• ' ' ,1 

r third axis. For the latter, the correlations suggest , that* certain observers 
may have had difficulty either recognizing or recording "student-student^ 
feedback in Spanish." Since the lowest correlations Involve Observer 2, 
it Is possible that this one individual is the problem'; however, none of. 
these correlations are as high as for the other axes. , / „ ' 

One problem is that this variable^- (as we mentioned earlier) occurs ^ 
relatively^ infrequently — particularly in the more* advanced classes, for 
this reason each such occurrence is highly weighted (as seen in Figur^^ .3.5),,^«!^^ 
and the failure to record any given instance could have a sizeable influence 
upon the outcome for that session. » • 

When correlations comparable to those given in Table 3. TO arj^ studied 
£ot all nine of the ordinal classroom interaction factors, a similar conclusion 
^>ls' reached; only -for "stjident-student feedback" is. there a serious problem, 
with ititer-observer c6rrefation. These results are briefly summarized in 
Table 3.11.^ ^ ' - . ' . 

Finally,' it can be pointed out that inter-robserver correlations for 
different scores were much lower than for the same score, indicating that the 
observers were indeed discriminating betw^el! different classropm interaction 
patterns. . • * • . * ' * • ^ 

^ Just a$ for the day-school data, we have plotted cSSissroom centroids on 
.the first two. canonical discriminant function axes ^ in Figure 3.7. Night-school 
teachers have been identified in the figure with, the letters' G through L and 
we 'have also included vectoi:s representing the original nine classroom 



Sotlce from Table 3.1 that observer 2 collect ed-^e^yy little data in the 
/study. . , . , ' . - • , 
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interaction factors/ * Interpretatio*ij. of this figure can proceed in the same 
manner as for Figure 3.2, Again, we find the night-school results to^ be less 



•intui-tively compelling thai! the day school results. 



Relation of Jeacher BaxzkTgrojand. Characteristics to Teaching Styles 

/ ■ ; . 

Considerable information w^s ol^tained about each teacher in bdth the . 
night and day-school samples through a questionnaire administered^ at the • 
outset of the study. It is of interest to see whether a teach^t's performance 
in the classroom might be related %o this prior infomnation about his or 
her educatiou, experience, and preference amt)ng teaching styles. 

' - ■ /. ■ • ■ • • • 

Only 12 teachers were studied, backgreund characteristics we^e often 
badly skewed (e.g., number of year^ of teaching ih Table 2.7), and information 
was available for more background variables than teachers. We therefore 
decided to use raj:her-^rude data reduction techniques in order to break these*^ 
data down to a minimal set of contrasts among teachers. . The outcome of this * 
crude approach to metric multidimensional scaling is presented in Figure 3.8. 
In this fi^re teachers are' represented by the^ same letters ^used in earlier 
figures (A-F for day school; G-L' for night school) ^nd their, background 
characteristics are represented as vectors. ' . ' - - ' 

Despite the approximate nature of th.e anai^ses under discussion, ^ 
Figure 3.8 reveals in^teresting .Information abdiit fhe teachers included in 
this study. The horizontal^axis is aligned with the variable", ^percent 
audio-lingual", and it marks a rather strofig contrast between day-school 

and night-school teachers. _Only one day-schopl teacher, /J, is* placed toward 

' ' ' »■ 

the "silent way" (left) pole of the horizontal axis. " ' , ^ 
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The* vertical axis is clearly an indication 'of educational level which 

differentiates *:eachefs within both the day and night-school samples.* "Years 

of teaching" is closely aligned witK this vertical axis but amount of 

experifertce teaching English ^s a sfecond language to adults ("ESL day adult") 

is aligned with ttie "silent way" pole of the horizontal axis. In other wopds,^ 

the more experience a t^^cher^has had teaching English as a second language 

to adults, the more lilXly he or siie is to use or approximate "the silent way," 

The more experience a teacher has had in teaching in the elementary or , / 
# ' " ^ ^' 

secondary schools,, the more likely 'they .are to use the audiolingual method or 

an approximation to it. It seems that, the lower the level, (from adult: to^ child) 

\ * ' ' ^ ' 

at '^/hich an individual Jias had experience teaching, the more likel)[ they are to 

be educated, experienced, and a'' practitioner of the audiolingual method 

■ \ ' ' ; - ' \ - ■ N ' 

(e.g., teacher K anrd L*). . . /-'^ 

'Comparing Figure* 3. 8. to Figure 3^2^ there 'appears to be some association 
between classroom interaction patterns (teachejr- performance,) and teacher 
• background chai;act eristics*. Notice that the day-school teachers- fall intp 
roughly .comparable circular patterns (from A through F) in Figures 3.8 and 3.2. 

. , THE REJJ^TION OF TEAc/hING PERFORMANCE TO STUDENT LEARNING 

• « 

' The-major problei? t>e 'solved in the analysis of the student ajchievement 
data was' .to deteniine 'hoi/^we could account for variation in ^posVt^st scores. 
There are three domains pfif variables whidh are potential predictors of po^ttest- 
scoyes: background character^-stics of the students, students' initial l6vel 
of proficiency as mea^re<i-^7 the pretests, ,and classroom experiences. 



Vatl^les describing the students \ backgrouncis are worth ^considering 

as pdtential 'predictors of f ifiaj. achievement because these measures may 'be 

indirect indicators of aptitude for learning, of academic skill, or of prior 

achievement of proficiency rin English* Obviously, the students' initial 

proficiency (th6 second doitfain, mentioned above) may^be assoi:iated with'* their 

final achievement status. Tfe- and post-instruction scores ^n the same test 

are usually highly correlate,d ,wLth each other because the experience acquf?ed 

in the interval which separates these measures does not great^ly 'alter the 

related order of students with i:espect to. their ^abilities; 'the latter hdve, 

of course, been built up over an entire lifetime of .experiences, for which 

the backgroimd measures ar« indicators or proxies. Neverthisless, the 

■ • - \ • — « « 

* ' . ■ " r ' 

relationship between pre- and posttest scores may be altered, through the 

* ' ' . ■ 1. \ ' ' . ^' 

J.nfluehce of intervening events, including classroom interaction experiences. 

Classroom interactions is. the third domain of predictor variables and Is of* ^ 

most interest* in this study becaus'e it is the' only domain over which some 



degree of ^control can be exercised* ' ^ ' . ^ * ^ ... v- 

^ther relevant experiences which might intervene between pre- and posttesting- 
(such ^s -use of . English at home, oXC the job, and in the commOnity) unfortunately 
could not be objectively measured in thi^s study. Tha, possibility of controlling . . 
4Siese extracurricular sources of experience is^^light ^itf any case, but it ' 

would be useful' from a theoretical point of- View to take th^m into account. 

- ' \ ' • . . ^ ^ r \ > ^ ' 

The best we can do. at. this point, however, is to bear. iiv mind tftafc^^ceftain - 

? ' ^ , * * . , ^ - ^ * ' ' ' ' , 

"background" characteristics might serve as proxies for sustained extra- 

-curricular experiences <e.g* , occupational l^il' for tfie n^ed^ to speak English - 
*in the workplace; fengttwo.f^tlme in the United States for assimilation into* . 
Ian ethnic community). ^ 
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Given 'these three Sojnaius of predictor variables, which are organized 

^\ ^ ' r 

in an obvious temporal, sequence (backgrojund experiences > pretest 

i • * * 

V * 1 * 

performance ^ classroom instruction), the analytic problem is to find 



haw the information cai) be most parsimoniously combined to^ predict final 
achievemer 



t. ! Multiple linear ^regression can be applied to this task, s^ince 



the squares multiply Cor;relation ) between a sdt of predictors and a' 
driterion variable indicates what proportion of .the observed variance in 

tKe criterion can b^ accounted for' by a given set of predix:tc^s. ^ 

• <? , ^ , # 

• Background characteristics must be taken into account fiifst because* 

/these variables .arfe, direct measures of or proxies for educational attainment, , 

* ' ' ' < ^ Z\ ' ' ' . ' ' ^ ' ^ ' . • 

competence to cope with rhe processes of schooling; motivation and aptitude. 

- ' \ . ^ ' ' ^ * ' " ' ^ V < ' ' 

If such variables account for most of the variance in final scores, then L 

classroom e5q>erienc^6.' can haye little" differ^titial influence on final status; \ 

. ' / " ^ • " \\ ' , ' ' ' *^ 

this is not to say that largie pre- to^osttest gains could not have occurred, 

only that they are'^.not likely to have changed the relative ordering of' students 

; ' ' ' . ' ' ' '* "^'^ . ' • * - 

,"in terms hf achievement. , ' ? t » ^ 

Next w^'must-don^Ser how much initial status in terms or'pretest scores , ^ 
adds to the prediction of outcomes, beyond' what we havei beeri^abl^ to learn 
from a knowledge of . background •characteristics alone. A related issue here 
'is howwell initial status per se <?an be pi^edicted from backgrpund characteristics. 
Thi^.^analysis^ teHs us . how adequate and useful our informatJLoii about, baqkground 
experiences is in the first place. * . 

The final step in the regression analysis is. to assess the unique Gontribution ' 

of classroom Experiences in tlie predict ion ?^of fih^l adhievement, above and beyond 

. " ? • * . ^> " ' ' - - 

any predictive utility. initial status and background^^experiences. We also 



must determine whether different forms of classroom ^interaction appear. to 
have different effects upon achievement. .The purpose of relating classroom 
interaction ^o achievecjeo^ is to find out how much this information adds to 
the prediction of achievement and to identify any components of classroom ^ 
•interaction which^can be hypothesized to have an impact upon specific forms 
of achievement. 

Notice tKat the. classroom experience variables differ fundamentally from 
all af the other variables in the analysis in that they are not inciividuli 

'•4 * ' " . . 

measures, but" apply to all individuals in each classrbom equally. From the 
point of view of this study, these, variables are the '"cLndependerft" variables 
whose possible effect upon final achievement we a^e most interested in learning. 
These variables take the form of contrasts among classrooms in terms of obserwd 
patterns of teacher-student interaction, the four transformed canonical vari^tes 
developed and discussed in Chapter Three. Every effort was made to arrive at 

t 

a small set of independe^it^variables which are reliable and not too higK^y 

intercorrelated, in line with the requirements of the multi^l^ regression 

i ■ • - * 

model. On the other hand; the background and pretest variables are more error 

t * ... 

prone; but they only .play the roX^e: af covariates to adj^t for preexisting 
differences among classrooms in the^ final analysis Moreover ,^ these dat^ are 
available for every individual, so. stable fe^tirbtes of the required regression 



pa"rameters can be obtained even with fallible measures. 

■ -7 i 



Predicting Fall and Spring, Student 

Achievement from Student Background Characteristics'- ' 

, 0 * * - - • 

% * • 

^ In Table 2.5 we. saw many significant zero-order correlations between 
\ student characteristics and student achievement scores, both in the fall 

and in the spring. The background and achievement domains are clearly related 



at the level of individual 'pairs of variables, so it is in order to * 
see how background information can be. optimally combined (weighted) to 
predict individual achievement scores and to determine* how^effective this 
prediction can^be when simultaneous use is made of all available background 
information. - . . 

• Each achievement test, score was predicted from the .background" characteristics 
using multiple linear regression. The results of these analyses for the day- 
schpol sample are^ Summarized in Table 4.1. In that table the labels across 
the tops of the -columns designate the fall and sprin'fe achievement scores; note 
that Ipgarithmic transformations of the oral proficiency scores were us6d» 
The rows of the table list particular background characteristics' used in the 

regression analyses. The number in any cell of this matrix i,s the standardized 

J* - . ^ • 

regression weight for a particular background variafble (row) as a predictor 
of a given achievement score ^column). These entries complement the zero-ordej 
correlations already given in Table 2.5. The former show the direct contribution 
of variation in each background characteristic to achievement, whereas the 
entries in Table 2.5 do not take into account the facf that the background 
characteristics are correlated. am6ng themselves. 

,The line in Table 4.1 labeled R ogives the squared multiple correlation,' 

^ . * 

-of each achievement test score with all of* the backgroundWariables, taken . 
simultaneously. These numbers indicate the proportion of the -parlance in eaqh 
achievement' score predicted by the complete set of background characteristic 
variables; for example, the background characteristic variables account far 
26 percent (.26) of ffie variance in the fall Decoding I scpres; fpr X3 
(.13) in the fall Decoding 2 Sco^^;^ Tor 44 percent (^44) of the variance ' 

;the John Test scores*, and so on. ^ ** ^^-^ 
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The F-ratids prfesentet^ in Table 4.1 ^degrees of freedom in parentheses) 
provide a statistical teVtVof t^he'null hypotHesis of ' no predictability of 

• . / . n : , ' • , • ' ' • - 

&cj} depend'ent Variable (Achievement Te.st score) from the set ot independent 

: ■ ^ \ " ' ■ • ■ , . ' 

variables (background va^iablfes),. In other words, is the mean score oh eacl 

-' '/ ' * \ 

test the best a-^ailaSie 'estimator, bf how any given student will perform, 
^or can we leatu more "about that performance by taking His or her biSi^oittr^/ ^ 
characteristics fnCo ^^ount? The probability levels associated W^th jtM^^ 
statistics are coded witli asterisks. 

' A prediction equation could be written with the regrefeion wfeights 
ifi any^ given .column of Table 4.1 as the coefficients ofMihe background' / 

variable3. The actual standard scores of a particjxlar studerit for e^6h of 

j . • - ^ > ... - / ' ^ y ^ ' 

thesGf variables woiild be 'entered i^ this equation fco predict his ,or her 
f *• \ . *r ' ' ' ' • ' 

standardized score on the cor^responciing achievement ti^t. It can be 

yarikbles .with have more influence/in the /predicted outcome. 

The ;f oilowirt^ example will illustrate this conc^t^- .l^We will use the .numbers 

in the column l^eled P-CT, the correctness^jicore on the Oral ■ Wqf iclency i ' - 

Test which was fake^in the spring.' The ^ediction -equation fot this scgr^ 

v " ^ * 

after logarithmic transformation is: 



sfeen that\ 



(1) 



^logU(P-CT) 



.0& Zg'-- .17 Z/ + . 



:fv,.^v . -.02^Z^Q+.0/z^+.U^Z^+.24Z, 



YE 



As Will be seen shortly, two/of the abovjs i^ights are signtf icantJo^ r 
^ifferent from zero in the statl^'tical sense. Tlik^ three which are uiXderllned 



/ 



in the Pt.CT column of Table .4.1 are large enoughs to give some feeling for 
which background factors might affect correctness scores (P-CT). They '*'6re \ 
age, which has a negative weight (- .17), status of lob in former country,/ 
X.'23) and amount' of English taken in the U. S. (.24). In other words, students 




wfto are yourger, had. better jobs in their, former country^ -^ii(|*have^ taken^ 
more course Lrork to learn English after arriving' in this douhtry will have/ 
higher predicted correctness scores. ' , ^ . i j • 

' There. a:e two other f eatures " bf .the information '^in Tkble 4.1 whl,ch 



should he no 



:ed.. Those regression coefficients whicfi ar^ ^fgnif icantjly 



different fri^m zero in a statistical sense are marked wifh aisterisks. In ' 
addition to significance in a, statistical sense, ^ however , it is important 
tp consider how much feaclv'predictor variable contributes t;o the^otal ' 



2 * ' 
predictability signified by R , without regard to sample'. size. The regression 

(^weights for thxise var^/rf6les wWLch contribute at least .01 to the mag^tude 
of the squared mumpie correlation, R^, have therefore been-^der lined in 

is account format least one percent of the 
\ ' total variance ).n the Achievement.Jest score), ^ ' * ' 

Itoplications of the Regressions oi Achievement ^1 • ^ * 
Test Scores on Background Characteristics , [ ^ ^ } 

' The R for .'prediction equation (1) is rather low (.20). Thi^ equation^ 

' i [ ^ 

is not a very accurate predictor of Oral Proficiency correctness sco|:es. We 

may still speculaTte, however, that, s^tjidents with three background character-' 

istics (Youth, Former Job, Eng. in U, S.) are likely to have higher correctness 

scores* in general as long as we remembet /that the prediction equation will be' 

\ > "N. • ^ . . r^' - • ' . f' - * 

; ' . • ' ' . ' ^ * " . . . 

> highly in error in any given instance. / • 

By way of. contralst^ consider the comprehension scor^es (P-CP). The 

' . 1 i / ' "--^ " 

prediction equation In this cas^.U likely to be a litfl^ more accui^.te 
XR .33 for comprehension sW ,20 fofe cotrectness). In that equation amount 
^ of pnglish stt^ied in i.t^ie'-'yi /S^ t&e 'aar|eVt>eight. Three, other Icharact^t- 

\ ' ,.-"'"'"1' ' hi' ' t : ' -. 4* ** " ' 

' ^|tics^re wOrrtK' noting: a^e/^ which' ^'ain has a negative weightv(- .,20), 



Table A.'l (i.e.,/ fflesT^vSri^ 



tS. ■ '!' stat«s"6£. 5ob' in former coii^tvy, ,(«28) , snd a©ount.-.of English in the I 
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S. <^34). 



Background characteristics predict comprehension scores liest (R « .33), 

2 ' ''/''.•' 

structure scores less well (R = .24) and correctness scores least accurately. 

A reasonable hypothesis from these data,* however, is that tHos'e students .w.ill 

be more.^ij^ficient in spoken English by the end of the ye^ who are younger, 

had a higher sit a tus job in their former country, and hav^ taken more English 

courses ^r programs since coming to the United States./ / 

It should.be remembere'd that predictive b^ckgroi^d characteristics do 

not necessa)::ily ''cause" higher or lower score^.. Age{ for example, does not 

necessarily make a person less proficient. But age is associated with several 

. ^ . / ( ^ 

other characteristics which, taken together, mav/^ive us some idea about why 

ag6 is a. negative predictor of proficiency. . Older/students' are more likely 

to have a low status jpb in this countiry, 6ence, may have less opportunity 

- ' " ' . ' ^1 ' r ' * 

to- speak English. 'They are more likely t6 Mye'Aad course work in English 
in the previous country than in the U. ^.^'/^hencie^ may have learned English 
.originally as a^school subject .rather ;t;1ian as something to be used in their 
daily life. • \ :' . 

^ ' ■ / ' ' ^ ^ - ' . 

Miount of English studied in this: country is more likely to be a direct 
influence on achieving proficiency. ,jSut, again, students who have had-'more- 

? 

work in English here are also yo.unget, have arrived more recently,, and have 

^ \ [' ' ^ )/ 

a history oT higher, level emplo3anent toth here and abroad. They probably 

have ^both fflore need to speak English and more opportunity to do so. ,Thus a , 

combination of previously acquired proficiency and opportunity to speak 

English is likely td f acilit£it?e . the progress of these students during their 

time at the Adult Learning Center^, .it 



Differential' Prediction of FalJ and Spring . ' • ' " '"'^ ^ 

Achievement' from Student BaclcRround Characteristics 

Qf...the five squared multiple correlations for predicting fall' tests from 

background variables, all but one are statistically significant at the .01 

level. Note also that the multiple correlations are generally lower in magnitude 

^ } * ' 

anli pf lower statistical significance for the Spring tests. These rjesults 

-suggest that the" role played by background experl^ences in achievement is 

V ' , - ' , . • . 

Substantially diminished over the course of ,the year, as new experience with 
English aS a second language is acquired through classroom instruction and* 
extracurricular act:^vities. That students gain substantially in their average 
tes£^ scores has already been seen. in TaBjLe 2.3* ' 

, lliat background' characteristics do help J predict performance in the fall 

2 ' ^ 2 V / * ' 

(R^-^ .44 for the John lest;, R =- ,.47 for jthe Literacy Test) but that ,t^eir 

% ^ ' i 2 ' r 

predictive utility is diminished in the,sprd$ng (R - .33 f.ar Proficieiicy- ' 

2 • 

Comprehension; R = .25 for Lite!;racy) is the most important piece of information 
to note' from Table 4.1. Only the cpraprehension score on the spring Proficiency 
Test retains a highly significant relationship to background- characteristics. 
This fall-spring distinction suggests that something beyond the students' 
background experiences prior to ins true tioi> accounts for their final spring 
achievement. Tfie next question is whether or not we can identify any component 
of classroom interac^:ion which accounts for the changfe in achievement. 

Note that the regression weights for "age'^-.are .generally negati:^ir ift 
Stgrt. In this cas^ we see weights which' are significantly different from 
zero in prediptihg both fall and spring achie^^emenf scores, ;just as ior 
former employment ie;vel and English studied* in the United States. 



It is otvidus from this ^alysis that a number of "background ch^act eristics 
are significant predictors of fall ^chievement an^ a few of them are of importance 
for predicting spring' scores. . Mos^ of the multiple correlations are significant 



when- all of the predictors ^re used in the, regression analysis, regardless of 
-.whether fall or spring achievement is Jthe criterion; , 



, Predicting Posttest Scores Using Pretest ^ 
. *\ Scores in Addition to Background Charaxrteristics . \ " ^ 

The next step in the analysis was td'prf^dict the posttest scores using 
inrformation ajjout the pretest scores, in addition to information about back- 
'ground characteristics. ''The. results of these analyses for the day-School 
'sample are presen.ted in Table 4.2* This^'^able is read in much the same way 
Tas Table 4.1. Acroslg the top, of the coiumns are listed the posttest scores, 
beginning with the^two Decoding scores, then tlie Literacy score, at/d the last 
three colunms contain logarithmic transformations of t'he three ^ores on .the 
Oral Proficiency Test. The predicted scores used. Aa this analysis were all 
scores/'takeh from tests administered in ^ the spring. - ' < 

Down tiie left-hand column .of the middle section of Table 4.2 ate listed " 
the labels for the pretest scores; included ate the two parts of t'he Decoding 
Test, the titeracy Test; the .John Testt, and\he Morano Test. The first line 
of thiSi^table gives the squared multiple oorrelations resulting^rpm prediction 
of ppsttesf^scdres from background characteristics; , these number^.T^re repeated* 

■ ■ .. - " •> ' ■ . ; . * - - - 

from the right-hand, portion of Table 4«1. The entries' in €he iine labeled, 

"R with Addition .of Pretests," &kn be'compared tcf th6 corresponding entries^ !" 

in th^ first liJie.^ Note iujdef D^^-S that whe^ only backgrouijd charact^istics 



2 85 - 



Standt'irdlsed^Ref^r^issdon Weights, Squared Multiple Correlations and 
Statistical Tests for Po^ttest Sepres Predicte^l frdmTretest Scores 
2\djusted for Bac^kgroand 'Characteristics; Day7S'bl?ool Sampli 



' L-S 



P-GT 



P-CP 



P-ST 



2" 

R from 
Background 



r-F. 

J-F 

*\ i'^^ M-F 



J 



.25 



.6A 
.08 ^ 
• -.97. 
.00^ 

.. .07 



-.08 
.2A 

.07 

« 

-.lA 



■ . .25 - 

-'.16"; 



.20 



.27, 



' 7 



.10 ,.V -.01 



.47 



39. 



.03- -'.OA-. 



.09 . • 



.OS 



.-.06 



;2A 

•it 

.25 
.01 
.07' 
•52' 

.11 



A:':. 



2*" 
R .with / 

Addition " 

of Pretests . 

F (5, ^66) 
, Test of 
Information 
increase 



.60 



11.17 



.19 



1.72 



..6L 



.72 



•*.82 



76 



12.23 2A.83 . 35.86 ' 27. 85, 



P <.-05 



P.< -01 



p < .001 



r 



V/ 



1 



9 mt 



Ik 



''Si 



ERIC 



309' 



2 

"are used is .25$ but R increases to • 6ft when the pretest informa:tiQnv 
is added. In other words^ if only background characteristics are^used to 
predict posttest scores, then 25 percent of the variance in the decoding 1 
pbsttest is accounted for. But an additional 35 percent Is ^accounted for 
if "inf o^pnation from thd pretest scores is added. Note, for example, that 
the R for prediafe±nr|^£tansfonned Oral Proficiency compfel\ensi.on scores 
f*ifom background characteristics is only .33, but \d.th the addition of pretest 
info^aat]^pn it becomes .82. A similar' pattern is. apparent for the other two 
^.scores pf the Oral Proht^ency Test. Thus, the students' initial proficiency 
plus some injtormation about their background characteristics accounta,_^pr 
^ sub^antial proportion of the' variance in the comprehension scores. In 

.summary, to understand how much the prediction is improved by ^ding pretest 

• ■ • • 2 • ^ ^ 

data po background information, simply compare the R in the first line with 

'2. 



the R 'in' the third portion of th^ table. ; 



It should be 'noted that most of the squared multiple correlations >f or 
^ . ■ * ^ - 

background characteristics plus pretest scores ate substantial (.60 to .82). 

* " ' ' ' 2 ' ' ' ' .W^ ' • 

Moreover, the. Increase in all of the S except for part two of .the Decoding 

Test are highly significant when compared to prediction from backgrouitd 

characteristics alone. * , ^ , t 

' * " * • • * * 

"Except for part two of the Decoding Test, all of the squared "multiple 

(Correlations of post test scores with background and pretest variables* are 

at or above .60. From Table 2.3 we* can see that the posttest reliabilities 

for all except Decoding 2^-*fan§e from .88 to .94 (Decoding 2^has low^r internal^^ 

consistency, yielding a reliah^lij^ of .77)* ^^It iife. take these reliabilities 

to mean ^that* roughly 90 perdent ^fTfhe variance in mast ppsttest scores. i& 

reliable, then it can be^said that over 60/90 or two-thirds or that variance 



can' be accounted foif by.the background and potest Varia^es .whicji we have 
measured ♦ This leaves up' to one-third of. th^ reliable postt^y' variance* 
which might be accounted for by cPLas^room experiences/ Howeyeir, V^e must- ; ' 
keep'in min<^hat we do nat .have 'separate measures of ^qlassroom experlence^^ 
-for each of the students. in any classrobm—only contrasts ^between classropmst . 
This means that we cannot.. fexpect to. increase the accut«cy of our .pi^ediction* 
to the limit, of the reliabilij:y of posttest measures on the basis of cLassroom 
experiences/, Eveji if cleslsroom Experiences are highly influential, their . 
effect cannot be detected unless there is sufficient fiomogenity among students 
within classrooms with respect to these: experiences. . 



Posttest Scores Predicted from' Classroom InferacCion, with. 
* Xdjustment for l^etest Scores and Background Characteri^stics 

• Table 4.3 presents information from the^regression analyses for the . * 

day-schooi sample in .which posttest scores were predicted froitf classroom 
. . " ^* ^ . , ' - . ' 

interaction contrasts, pretest scores, and background characteri^ics. This - 

table is organized in the same way as Table 4.2.^ 

2 

The firs.t line of Table 4.3 gives R fron^ the regression of posttest . 
scores on background characteristics and pretest scores, as .seen' already in 
T^ible'A.^. Next are four lines with Roman numerals^ I through IV; within 
* each line can be found the standardized regression ^^weight of the corresponding 
'transformed canonical variate^tthe four "higher order'** bipolar * contrasts^ 
descrilDed earlier and in Chapter Three). ^ . ' - ' 

^The sixth line ill Table -4.3 gives R^once the 'four transformed canonical* 
^Wiate classrooift infbKractlon contrasts have been included in each prediction 



i ■ 



^ - 28a- - 



'table 4.3 



Standardized Regre's(^ijon'Wifeights> Squared Multiple Cprrelations^j^^ 
. \arid Statistical Significance Tests fot-Posttest ScoresL Predicted^ 
.f&ra, Classroom Interaction bpntrasts, Adjusted for Stud'ents Background 
\ .Characteristics and Protest. Scores; Day-School Sample- *• 
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equation. Again, the lines containittg. the initial and^final valued can 

be compared. From thi^ comparison^tje cah see that ti]B addition of classroom 

(interaction* contrasts adds littj.e in an absoluti^ sense to the accuracy of 

prediction of pasttest scores> (iticreases in R va^y from^ .04 to •09).n ^^-^ 

Despite the relatively small absolute increases in the a,ccur^cy of the 

r / [ ' • ' ' * - • ' "'^ 

fSr^iction^ obtained by supplementing back^totmd and pretest information 

^^Vlth classroQpi interaction information, it can be-^seen from'^the F-test results 

in Table 4.3 that the increases in predictability are st^tisticaily siginif i'cant 

As we pointed out earlier, the classroom interaction contrasts apply ' ; . 

equally^ tfo all individuals within , each classroom, so. great homogeneity, oi^* 

experfencee would be required in. order for these contrasts to yield a large 

absolute 'increase in the accuracy of predioti^nr-of^asLtest 'achiievemmt^ , The 

* • . » • , ' o , 

F-test results relate, however, to relative increases fiTNaccuracy of prediction 
*• . • . ' """""" ^ ' 

These relative increases in the accuracy of predittion are Substantial enough, 
even given our smalF sample size,, to support further exploration of hypotheses 
ab6ut classroom interaction-.^ ^ ' % . ' ' , ^* 

Recall that the first tvfo classroom ;.interaction contrasts reflect ,^oad ' 

' * - ' ^. ' ^ 

styli>stic differences among teachers.^ The negative, pole .of each contrast ,^ 

- ' ^ " . • ■ ^\ * * ^ 

relates^A .more highly structured^ class or .^oup-rfriented activities. The 

prevalence of "tether model-student practice"" tnteradtion patterns on the, 

negative poles^ suggests thaf a. variant of the audiolingual method was being 
- \ ' . / ^ ' r • . 

used. But, the "Free" and "Other" pa^ttems of ' classroom interaction on the • 

positive pole of the second transformed axis suggests that this varlate is a 

contrast between the silent way and audiolingual methods. The ^flexible, 

individualized; supportive, and. Encouraging aspects, of th^osit:Jye pole ' 

of the first axis suggests an eclectic style of classroom ir^teractio)i. 



The third axis. relates most directly to the occurrence of ' "student-student 
feedback" during instruction. The fourth axis can be '^ignored for the momeht. 
• The most striking aspect of the pattern of standardized regression 
coefficients presented in Table 4,3 is that it appears that those features ^ ' 
of cias^xoom interaction which are associated with higlier than would other- 

^ wise be* expected posttest scores^ on the Oral Proficiently Test (i.e., axes 
I and II) are .associated with lower than would otherwise be expected posttfest^ 
scores dn the Literacy and Decoding Tests (compare the weights in tbe " 
left-hand columns of T^ble 4. ^3 pg Eg^te I and II with those in the rights 
h^d columhs opposite I and II). If- ve were io take these 'coefficients 
as t?iie basis for formulating hypotheses about the effectiveness of individual—^ 
ized iastruction vs. group instruction, the silent way vs. the audiolingual 
method, an open and supportive vs. a highly organized and directive, classroom 
climate, then we should expect. the firmer (individualized, supportive) silent- 
way) to lead to increased oral proficiency while expectirrg the latter (grouped, 

' directive^ audiolingual) to l^ad to increased. literacy and decoding skills. 

It appears, therefore, that there i^s a tr^de-off in term's l)f the achievement 
g^9als — what appears to -be incremental for oral proficiency appear^s to be 
detrimental for litefacy^ and decoding, and vice versa . The same can be said 
for axis III, ^'^t'ude^it-studeiat feedback," since its occurrence in a classroom 
is predictive -of lower th'an might otherwise be e3$ec1:ed oral groflciency t * 

^ / ' , ' ' , . * ' - r 

(mainly comprehension) but-high^ than might otherwise be expecteS literacy 
^and decoding^/ ■ ' • • * . • ' . . ^ 

As for that ubiqaitous bipolar contrast in classroom interaction patterns, 
"teacher direct-student read and/or ask (Question" vs. "teacher question-student ' 
ansv/er," wla "again see it playing an important role. In this cas^ there is'^ 
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.no apparent tradeoff, however. On the contrary, the data suggest quite - 

i » ' ' ' • - ^ . 

^ clearly that those classrooms in which students are directed to read are ^, 
'^aiso those classrooms in which we ban predict that" the students will axrhieve 
.higher than would otljierwise be expected on* the Literacy *and Decoding posttestS. 
The contrast seems to bear no real relationship to oral proficiency, suggesting 
that nothing is to be lost and literacy and decoding skills may well be gained 
by using* the, "teacher direct-student read and/or ask questions" paradigm 
• (line. labeled IV ih Table 4.3% ' * u • ' 

The. Analysis of Differences in Student Achievement by Classfooms^^ * 

We can now carry the process one step further than was done with the ^ 
data as presented * in Jable 4.3, by allowing parameter estimates for an^ possible 
differences 'between classrooms- or teachei?6. Thus we no longer restrict ourselve'a 
' to the four classroom contrasts derived from observations of teacher arid .student 
interactions oyer ,the opurse of the study ♦ ^ . v . 

- Predicting Postt;est .Scores TJsing All Possible. 
'* Contrasts Amdag Classroom^ with Adjustment for 

Student Backgroiuod Characteristics and Pretest Scores . 

In the first line of^able 4.4 we have presented again the squared multiple 
..correlations of posttest variables, with background and pretest sc6res. TK'ese 
entries are the same as those seen on 'the first line o£ TaM^ 4.3. The second 
linl of Table 4.4 'contains comparable values. These R^'s Vere obtained by , 
entering" a complete set^of..l3 contrasts (onfe .for each degree of free'dom)- 
jmong classrooms into-the prediction equation. The F-tests of information- - 
' increase associated with, these R^ values are %ir statistically significant, _ 
indicating tha£"there are re^ in student £&sttg8t a^^^ - 

among - the various classrooms in the study . • - . 
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Regression Analysis Hesults when Possible Clas^raom Effected'" are Entered; 
Compared to Results when only Four Classroom Interaction ^Contrasts are Entered; 
^ Day-5chool Sample' . ^ 
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/ -rfie l?st four I'ines iti Table 4.4 cont&in infD.riiiatfon,abput the separate . 
contributibns. of out original ftSur classroom interaction contrasts Ctraifisforme^' . 
•canonical variates)',, as well as the ^CT^ble remaining classrooC contrasts, 
to" variation in-adjusted cl'assxoo/means 6n the respective posttests. ; 

From the last four lines 6fc Table 4.4 we can learn a 'good deal about 
the possible ba^es of differences among dlassrooms on posttest -performance, . 
beyond what is acc6unted for by; the background characteristics ,and prefce.st .. 
perfomance of the students invplved. . Ortly for the Decoding 1 posttest is . 
±t safe to conclude that the four c],assropm interaction )contrastS of the 
classroom interaction . datS are sufficient to, account for all classroom 
differences in- adjusted posttist performance, - • . 

Erom the last two lines of Table 4.4 we can see that^the Literacy, 

Proficiency Structure, Decoding 2, and, perhaps^ Proficiency Correctness 
^posttests all show class^om-to'-classroom variation in adjus|;|d means which 

must be attributed to something beyond''the reliably measured aspects of 

classroom interaction embodied -in the four tran'sfopned caiwnical 'variates. . 

This fact provides justification, for lodking at mean performance data " for the ^ ' 

'individual classrooms in order .to seefc an explanation f6r their differences^ , 

^ < ^ ' . ^ ^ ' . . , . ^ 

Simple Comparison of Pretest and ^ . ^\'[ . v.: 

^ ' Posttest Performance in Day-School Classrooms ' / 

In order to study the mean performance <-of individual classrooms we reverted 
to simple, analysis of covarian'ce procedures in which each pretest ^as the sale 
cpvariate for its respective posttest. Ari^p^on;- in the case^of the pral ' 
Proficiency Test, only the correctness score was investigied and both thfe 
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Decoding 1 and/the John Test were considered', -individually, as covariates. 
For the sake of completeness, the Decoding 1 Tests were also studied witH * 
Isimple analysis of covariance, even though there is no 'evidence in Table 4»4 
*for classrooni ""differences beyond the four classroom interaction contrasts. 



Figures 4.1 through 4.5 are plots of the 14 day-school classroom means 
on pretest scores (horizontal axis) vs. posttest scores (vertical, axis) for; . 
the Decoding 1, Decoding 2, Literacy,, and Proficiency Test^ (the proficiency 
means are in the log^Q uAits and are plotted against John Test^cores in' 
Figure 4#4 and fall Decoding 1 scores in Figure 4.5 because the P'raficiency 
Test had not been' administered as a pretest). " Individual classrooms are . 
coded in the figures by the same letter ancf number sysjtem used in 'earlier 
figures. ^ 

The litie drawn through 'the points ih each figure is the pooled-xdLthin — 
classrooms' regre's^ion line, and may be talcen as the point of reference for 
determining the adjusted gains or losses of indiy;Ldual classrooms. Th*e>way 

^ : ^ " ^ ^ ; ' * 

to use this ffgure is as follows' (refer to Figure 4.4): : ^ ; * 

1. Along the horizontal axis are units of scores* on the John Xest. ^ 
j:'^sume that a class had a score of 20 on the John Test in the fall; 
. 2. From this point, draw a vertical dine to the regression lifie, and 
from where this vertical , line ^aeets tjie regrete^ion line draw a 
' horizontal line to^thd Proficiency Scrore axis. When this procedure 
is followied, the horizontal line from the regression line would 
intersect the v«j?tical axis at about .75. <* 
3. This score (.75) is the predicted score for classes whose mean 
• score on the John Test in the fal]/^was about 20. ^ > 




19- 
18- 
17- 
.16- 
15- 
14^ 
13" 



Class 


Size 


AI 


5 


Bl 


' U 


B2 


1 


CI 


3 


C3 


^ 3 


D2a 


7 


D2b 


U 


D3 


6 


E2\ 


10 


E3a 


la 


E3b 


10 


Fla. 


7 


Fib 


• 4 


Flc 


1 



■/ 



1 1 — ^ 1 1 1 n r-rr-i^ 1 r ( t;^ I I I I 

13 14 V15 16 17 18 19 20 ^ 21 22 ,-23^ 24 25 , 26 27 -. 28 29 30.^,31 
- - ' ' " " ■ ^ PRETIIST 



FIGURE 4.1 



• . pecoding 1 Classrooni, Means for Fall and Spring Aaministrationsi 

' Day-School Sample "(Poolea WithiivClassifooins r = ,69; Total |a«ie r = .73) 
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FIGURE 4.2 



Decoding 2 Classroom Means ifor Fall and Spring Administratiprisj 
Day-School Sample (PooleW Within-Classrooms r = .48; Total Sample r ^ .27) 
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IGURE 4.3 



Lliteracy Classroom. Means for Fall and Spring Administrations,^ 
Day-School Sample (Pooled Wfthiij-Classreoms r = .59; Total Sample r = .71) - 
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FIGURE 4.4 



^^fe^lQt^P^^^^ Prof icieiicy' Correctness Classroom Mqan 
Scores x Fall Jolin Test Classroom Mean- Scores^ Day-Scho6i ^ 
Sample (Pooled Within-Groups r = .2-8, Total Sample r « .77), 
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Thus,* points oV the regresgian lirie Tepresent predicted, spring 

scores on the test .belngs^rfrisiderfed* \ , 

5» The poJLnt^^in the f igure ^fenid the regression line represent 

the actual scores, A.l, for exaiaple, has ^fallen above the iine^ 

Its -actual spring score^ is bette^ than would* be predipted on the 

basi$ of ^the fall John Test. meAi for this class. . - 

6, As can be seen,, some classes fall above the line (F.lc, B.l, C,3 and 

others); others*. fall below- (Fi,la, F.lh and others). 

These fjLgures may be used in several different ways: (1). to ' 

compare the same class on different measures; (2) to compare 

different classes of the same teacher; and (3) to compare ^the , 

^ performances of classes at 'different ^.Ifevels of prof icAencj;. Us4.ng ^ 

Figures* 4.3 and 4.4, compare the relative positions of F.lc on 

. tJie Literacy. and Proficiency correctness measures. This class' 
^ , . ^ \'' 

is doing better than predicted on the Proficiency measure and , 

poorer on the Literacy 'measure. ^Now note in Figure 4.4 the • 

three'^lasses of teacher two are doing poorer than predicted 

(F.la and F.lb) and ona- better than predicted .<Filc). Ag^in 

referrdJig t6_ Figure 4.4- we-see-that-47)f 'tTTe15'Le ^ 

. * / are doing better than predicted; 3 put of 4 of the Level 2' class.es 

'■/JV and all of the Level- 3 classes are doing as well as or better /tK^n 
^-^"^ ^ " ' , . ' ' ' . - / • 



. predicted. 

• Differ ences In Achievement pf Literacy ^ \ / . . - • 

- . " Referring back to Tabll 4.4, we 'see tHat tbe^ Literacy Test slt^^s evidence 
? ' of mucfi lariatioA which cannot: be accounted for by ifeference sol^y to the ' . , 
' • :'. fou^' reliable classroom interaction contrasts.. From Figure 4.3 we see'^tha^ ■ • _ 
tr' ^his yar^?:io»r:i:8 probably due tolh^lfixcept|9nally good posttest performance ; 
r^ERlGf .th6„level 1 students in classroom CI. thlTe students begiq^ t^e year'at . • 



4 



.^the very lowest level of performance on tTie Literacy Test, but,, by the end - 
^ af» the year, they are as literate as students in several level/ 2 and level 
3 classrooms. This performance contrast^markedly with that of students 
111 classroom Bl, where spring Literacy Test perf ormance' is essentially 
unchanged from that in tbe. f all. Obvibusly, teacher^ C is increasing .the ^ 
English language literacy 6f J.evel 1 studet^ts more th^n other teachers ^t« 

\ ^ • . ■ ^ ^ ' ^ ' ^ ' 

' t;hat level. A similar conclu^oil holds.-for Decoding 1 in Figure- 4.1. . , 

Dif fere_nt:es in Achievement of Decoding .Skills ^ ^ ' 

"'^--tTie'Decoding 2 test has b§en revealed to be .a bit anomalous in the foregoing 

analyses -in that performance^ on the test is not highly predictable from a 

knowledge of student -background. characterlstics^,_Rretjgstj)erf^ 

classroom interaction- experiences. Moreover, there* ar§ re'al differences . 
•*... , . ■ ■ - " . - 

among classrooms in teijns of Decoding 2 scores which cannot be accounted for' 

" ft * ' ' ' * , 

by reliable differences in observed classroom interaction patterns. Figure 4.2 
helps clarify what might be going 6n sd^h Decoding 2 scores, which in part * ^ 
measures how well students can identify sounds a^d words in English. It seems • 
- that certain classrooms in.which Students have the poorest relative gra|^ 
\of phbneme-gralJheme correspondences in the fall are the very classrooms 

in which the students have achieved^ rfela'tively superior, -g^asp of these 
» correspondences .'by the end of the^^sch^l year; and vice versa. One obvious' 
way in which we can interpret. tft^se results is tie.;hjrpoth-fesize that teachers 
■ ^in some, gense overreach to their students' initial; abilifies in English 
.^onunciationV ^if these abilities are itiitiaUy minimal? the improvement' , 
is* sought; if. these abilities are ini^tlally superior, then other aspects 
of "performance are emphasized. , ] 
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' - - .m 
Differenc es In Achievement of Oral Proficiency ^ . - , , * '. 

As for the Oral' Proficiency Test structure scor^ little Is reveled 
by .inspection of Figures 4.4 and 4.5 except that, the achievement of level. 1 
students in classes Fla and Fib Is lower -than might be' expected. Notice, 
however, ,^^that neither the^ John Test nor the Decoding 1 Test Is an'optlmip 
"pretest'^Tfor oral proficiency. . ^ 



• Between Classrooms Relationships Among Sfeudent Background 
- Characteristics, Achievement, and Teacher/Studeht, Interaction Variables 



A comparison of 'Table 4.1 with the corresponding values in'Table.4.5 
reveals eubstantiar change in the predictive role playeci by age as well as 
history of .employment In ^he country o<^ origin. .This suggests that classrooms 
are differentiated not orfly by level of proficiency, as measured by initial 
John^Test sj:orefS7::4ut by certain student '^kgrourfd cha^racterlstics as well. 
In order ^to explore this possibility ^we turned to univariate atvalysis-of 
variaflfee and^canci^ical discriminant function analysis. ^ 

^ Not too surprisingly, one highly significant discriminant axis emerged 
from the analysis, §n^ it a^xlJunted for '59 percent of tT^e between-classroom" 

^ / ^ . * ' ' ■ * \ ^ * ^ 

variation. Details of the analysis need not "be presented here; suffice it", 
to' say that amount <i English studied In tfie U. Sk, English studied in th^^ 
previous count.ry,^;Lenlgth of time iVtbe U. S., years of education,-' age (younger), 
abd previous occlipaticJnal level were weighted ln..tha first disci(^Mnant^- j • 
function in the order Msted. "By taking into apcount the second canonical 
axis, tdie i^ercentage of yirlaSee afeckunted for could be^brought up to> 73 
percent, and the resulting solution is plotted in Figure 4. ft. The second ^ 
axis Is of marginal statistical significance^ it is a Weighted linear combinatioh> 
of sex, English studied in the previous .country, » and national origin^^ in jthat 
order. , . ^ * ^ 
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^Aa- for other figures relating tc^ discriminant function analysis, we have 

represented' the classroom centroids in Figure 4.6 with "unique letter-number 

coded, while the student background char'acteristids have been represented 

. ■ '• , . , - ' . • . ' 

by, vectors. Inspection of the figure reveals wide variation among day-school 

classrooms in terms of student background characteristics. It can also be 

seen that much of the variation among classrooms is related to level differences 

as determined by John Test scores. 

The background characteristic vectors are^^lso plot'ted in Figure 4. 6* 
There seem to be some rather suggestive clusters: youth and amount of English 
studied in the U. S*; educational and occupational leyel in the previous 
country; amount of English studied in the previous country and occupational 
.level in this country; sex (female), "origin (Cuban), arid length of time in . 
the United States. All in.al,l,' these results suggest the possibility that , 
Students have been assigned to mote-or-lfess' homogeneous classrooms on the 
basis of background characteristics in addition to their John Test scores* 

It is enliglitening to compare Figure 4.6 to the earlier figures in this 

chapter, in which pre- and posttest achievement are plotted without faking 

. * f. -> , '-' 

any notice of student background characteri^ics within the classrooms 

* < *' ^ , 

Involved. This comparison makes the exceptionaL Literacy gains of students*. 

in classroom CI seem all the more impressive, for instance (Figure 4.3). 

When Figure 4.6 is compared to Figute 3.2~a similar analysis of 

classroom differences,' but in terms of teacher and student interaction 

patterns— -seyeral fascinating points of agreement emerge. ' Notice .that the • 

two classrooms taught by teacher C are widely discrepant with regard to * 

student background characteyistifcs (Figure 4.6), and that these t^o class- 

^ *' * * - ■ 

^rooms are taught in markedly different ways by teacher C (figure .3*2). 

t - ' ^ ' \ 

f ,M ' ' > ' * ' " • 



These data suggest that""the mannef 'In which a parti^iular class is taught 

may reflect a' special adaptation on the part of the teacher to the background* 
• ^ » * 

characteristics and ability level' of .the students involved* . . . 

Although ^ hit more speculative than our reasoning with re-gard to 
teacher C's behavior, nl^ice' that" the three level 1 classrooms taught by ^ 
teacher F ^are spread widely with regard to background characteristics - 
(Figure. 4^ 6) and that their respective locations, in Figure 3.2 suggest the 
•• application of a more directive teaching strategy in. the less well prepared 
Hbfeiassrooms. As for teachers B and D, there seems to be a possible assqciation 
between "^the amount of student-student feedback in' Spanish which occurs in a 
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classroom.. (Figure 3>^) and its composition in terms'of cbi^try of 'origi 
andl^ngth of ti^^n.,tie Uj S/ (vertickl direction in Figure 4.6); a similar 
inference can- be drawn xor teachej C. -^^ 

In summary, we have accumulated cotisiderable evidence suggestive of 
relationships between, classroom interaction variables and student achievement 
variables. . The regression analyses presented in earlier sect;ions of this 
chapter were designed to reveal how classroom interaction' experiences might 
come to affect student performance. Nbw we''see, however,' that student 
performance and. background characteristics -may well affect classroom interaction 
that? isf, a model o'f reciprocal causation is probably most appropriate for these 
• data. , 

The Structure of Between Classroom 
Variation in the Adult Learning Center 

'■ The canonical correUtion's feporfced ' above give evidence of rather strong 

* ' . • < ■'■ ^ , . . . ■ _ • 

relationships between student background/performance characteristics a^d 

teacii^r/studpnt interaction factors at the classroom lev4l. It is. in order ^ ' 
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to ask whether we might be aUe tj^ find 'these relationships back^down-at 

# . \ ' » - 

the leA^el of individual background characteristics, test scores, and class- 

room interaction factors. The ari^er is yes, although we' must keep in mind 

that the results arg dependent upon /5nly 14 classrooms ^studied * in the Adult 

Learning Center and may not generalize widely. Only whe^re a ^similar^ system 

. of assignment :of students to -classrooms, a similar range .in teaching .styles,^ 
and a simil^ar population of students can be assumed to exist will these * 
results ^generalize to other ESL systems. . * 

In this analysis all the information obtained was used in one analysis: 
students' background characteristics, their pretest scores, the factor scores 
for .patterns of classroom interaction, and posttest scores, x The purpose of 
this analysis is to see if there were 4istinctive patterns of these domains 
of variables. The results are described by four factors in terms of which 
the classes may ''be differentiated by- background, classroom interaction patterns 
pretest performance* and final achievement. . 

Tfe found a tendtocy for superior Adult Learning Center classrooms to 
contain students who have a history of higher than average English Study"*" 
in the U. S. , Level of ^rmer Job, and Educational" Level; while having lower 
than average Age* and Level $f Job in the U. S\ We found * an association of * 
thesfe. characteristics with superior performance on all initial achievement 

.'measures • ^ * 

These ^^tuderits weire tau|i^t xd,th the. "free response", mode of interaction, 
as might seem appropriate, for students of higher than average initial dlbility*- 
"Student-student feedback" either did not ocj^r or was not allowed;*nQr was ^ " 
the model-^pr act ice-corrective f eedback-model-rpractice" paradigm used. There , 



' ' '. 

is a tendency for students within classrooms with such a, high j-level of ^ 
prior training and initial, ability not to be expected to read and/or ask 
questions under direction -of the^ teacher; on the contrary, a (group?) 
question-answer interaction pattern goes along with the "free i^esponse" format 
of instruction. • , * , * 

But in these classes -somewhat above average Oral Proficiency "correctness is 
associated \d.th. below average Decoding 2 ^performance. Here we see again the^ 
phenomenon noted in connection with Figure 4.2; students in higher-level 
classrooms appear not to be exposed to instruction in phoneme-grapheme 
correspondence — they are consistently surpassed in these skills b^ students 
in the initially lower-level classes^. * ^ . " , 

There was a tendency for some 'superior .classroomis tQ contain students 
who have been in this country for a relatively ^ng. while and who are , 
established in higher level jobs. They also have a history of English study , 
in the former country VWch supports our earlier conjecture that prior English 
language training may. be instrumental in the acquisition of hi'gher level 
employment in this Qountr^. These classes have exceptional achievement of 
literacy along with*an above-average achievement of proficiency. 

' One ^classroom interaction pattern characteristic of these classes is the 
"direct-read and/or ask questions" interaction factor which is highly 
associated with posttest literacy. Other characteris^tic cl^s^room interaction^ 
patterns are "Other" arid "Free," along with the "question-answer-corrective 
f^eedbacTc-prompt-answer"' sequence., 

• A third groap of superior classrooms contains well ^employed females who 
have been relatively well ;edtic%ed.'' Individuals in thes^ classrooms, who 
may well have arrivcd-frem^yba some time ago (relatively speaking) , have had - 
Superior 'English, language training itx their douhtry of origin*, but not 
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sarily in ,the United States. 



Above average performance' on the Morano pretest characterizes this 
group. The Morano sqpre is an indicator of granmatical skill. The Oral ' 
Proficiency' correctness score als« has its i^ighest loading on this dimension 
as do the posttest' decoding skills. This substantial achievement could well ■ 
be attributed ^to the high current as well as pkor level of employment, 
former ESL training, educational level, or sex (female) of ^t^ihdividuals 
involved. ' ^- . - 

Two interaction patterns predominate in ^hese classes:" the !,'Other" 
^factor and the "teacher-direct-student read and/or a^.qxiestion" factor. ; 
Oth^r classroom interaction patterns are deemphasized, including "model- i 
practice" as well a? "question-answer-corrective fe^dback-prompt-answer, " 
.The fourth factor is strongly related to 'all but one of the original 
nine classroom in^etaction factors. The pattern of loadings suggest^ the 
familiar contrast between classroom interaction factors, seen along the . 
horizontal axis of Figure 3^: QA-IND, CF-P-A, SFBK, FREE VS. MOD-.PRAC," 
CF-M-P, DIR READ, and OTHER. ■ At earlier points in our discussion we h^ve 
referred to the positive (left in Figure 3.2) pole as flexible, free,' supporti^, 
eclectic, individualized while deferring to the negative pole as structured, 
class or 'group oriented. \ - , ' . 

Those classrooms in which a more supportive and individualized pat terrf 
of teacher-student interaction prevails tenci to contain students who are recent 
arrivals from Western Europe, perhaps' not too highly educ^ted^ a bit younger 
than average, -and male. This means, of course, that the more highly structured 

.classrooms (e.g., those of ' teacher. in Figure 3.2) tend to contain older 
females of Cuban or Cafribean origin who^have been in the United Sjtates for 

•^ome jtlme* . ^ ^ \ * . - . ' ' 
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As for test performande, it is clear that the recent arrivals from 
Wester.n Europe are very deficient in Decoding 2 skills- (phoneme-grapheme 
correspondences) in the fall but become exceptionally' proficieht by thg,..^ 
end of the school year. I ^ . * ^ ^ % ^ 

What, these four factors of between-classrooms variation reveal ds the 
comprexlty. and richness pf the ESL training in terms of student background ' ^ 
characteristics, initial t^t performance, student teach.er interaction, - 
patterns, and final achievement. The Adult Learning Cent€^r must be'viej^ed 
as a dynamic system in which students' are channeled into classrooms which 
promise to provide them with an opt'ijnal learning 'expetlerice. There are dt 
least four ways in ^l^hich classrooms can come to be above average in the 
achievement of English language proficiency at the end of the* period of . . 
'instruction. It is clear that initial proficiency as determined by;the- 
,John Test is a good across-the-board guarantee of final proficiency, not 
' surprisingly;, but, among' other aspects of ^pretest performance, background 
characteristics and classroom interaction patterns must beytak^n into account^ 
in order jto'^ characterize fully the variety of patterns of achievement seen 
^ in different classrooms. ' ' 



c CONCLUSIONS MD RECOMMENDATIONS - ' 

• ■* " ^ 

The^'sitigle most important analysis in this study revealed the interactions 

^ among students* characteristics, their initial proficiency, classroom interaction 

: 4)attems., and final achiey.emerit. Some classes performed better than others. 

- These classes fell into three distinct 'groups in which different methods of 

'(^^ "iristrvietion ver^e^iised. . ^ \ 

... • . . . • . • . 
I d ' '^i.- . ^' • . . ■ - 334 ^ ' ' . • -. - 
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. Before we comment on these arrangements, it is important to" point * * ' 

* ' ' ' . ' ^ f ' 

oiit that we. are discussing differential or relative achievement and not - ' 

• absolute achievement. All of the classes made gains in proficiency, gome, 
however, 'made grekter : gains than would have been predicted from' their ' ' 

. itnitial scor^. were able: to differentiate among these clas,seg in terms' 
•• '■ of t^e students' characteristics, -their initial proficiency and "the classroom 
» patterns of interaction. ' ■ ' v • 

/ ^Oae type of class' of above average" achievement was typically composed of " 
younge'r students, who" had attained- a hi^er educational 16vel,' had st'udied 
more English in the" Unified States, and who, in their native country had held 
. Higher status jobs. This type of class had higher than aVerage Oral . 
Proficienfcy correctness" scores', but lower scores .on. the measure of phonic • 
/ skills (Decoding 2). They were largely taught .in. the "fre.e. response" mode 

which meant that the students were encouraged to generate Engiish.statements. - 

' ,. ° - • 

The interaction pattern is characterized by the teacher asking 'a question and 
. ' . V*- . „ ■ - . * ' - . _ - ^ 

the student answering or also asking, questions. ■ " . - " 

. : _ ^ .• t 

A second type of ,class was- coinposed^'ttf "students who had" been in this 
' country. for a longer time, who had studied more English in their native ' * " 

• country, and held* higher I'eyel jpbs,.in this coOntryC These students achieved 
• j^beitter than average proficiency and perfprmed exceptionally well on the 

literacy measure. They were taught largely in the^irect-re^d and/or ask', 
•questions",' pattejn in which the. feacher directs the, stydents to read some- 
■'. ^ thing and ask questions about , it^r the students"ask ^^stions. The teacl^r . 

.\works from'^a set' of materials that form the basis 'W asking questions. 
■'■ The teachers of these classes also usedS|he, "Other'\and "Freevresponse"' 



ahd the "Question-answer-corrective feedback-prompt-answer*' modes. These ^ 
' variations, reflect the ^use of different strategies within a tiontext of • 
talking^^iabbut materials* , , V. : ' . i 

ERIC -.^ ■ ..; ' •; ~ ■■• ^- ^ 



^ '\ A thk*rd***group ^as composed ol females wh3 bad higher status* jobSy' 
who were trelatively.bet'ter educated; |w6re from Cuba ,^J[ja(y been here'ibnger, 
and who had studied more English in thfeir'- native co\m^. '.^Their performa^^ 
was superior on all measures that required correct usage—the Oral Proficiency 
correctness scale ^ tlie Morano, and the Decoding tests; They were taught 
largely in the ''Other" mode and the "Teacher directr-student read and/or ask* 



question" inodeff* 



^ These i^fferences suggest a hypothesis. Assume that some, teachers had ^ 
diosen the appropriate met^iods for" the type of student. Then /the ^)p"thesis 
1$ that proficiency is increased to the degree that the appropriate method 
is chosen .^pr t^e t!ype.*.o#»"studrent. While *his conclusion .is hardly ^startling, 
guides for practical actipif\re appax^nt* in the«data. . The thr^e types 
described, above cfomprise the' major^y of students in the Adult Learning 
t;§nter. ' Presumably the ea'fri-est<*way to adapt methods to types o^ student^ 
is toljrgknifce cljg^sea jLn tirm^^^the sl^ents' characteristics and ;to%h2 
teachers' ufee the methods.. ap^?®p^pte to ^the type of ^lass. 



lave 



The tenter 



presently pjaces sEudentp in ^clasWes^on the basis of ^their 




i^ievel of prof ic^Lency .as measured "Sy' the John -Testis It, is important to note • 
^^ 2^^ alone deterfaines how 

nmch additional proficiency is achieved .', ' if ^pTpptiate methods are used 
In relation -to 'the students' 9har&tetistics^,. greater than expected proficiency 



is achieveti^ Jfe recommend, therefore: 



1., That students be placed in classes Wn terms, of their • 
initial proficiency- and that they -also be grouped within 
levels of proficiency,, as much as^p^sible, ]by common 



b^^jSfound-lsliaracteristtcs j and in cluster^ like those 
'"described above.- % ? 



2» That teachers be assigned to these classes-%whose teaching 



style is appropriate for 'the type of student. 
This second point needs some additional explanation* The variations 
on the' audiolUngual method have limited effectiveness for the types of 
students attending the Center. In some of the first analyses the interactions 



characteristic of this method were associated with greater achievement. But, 
' when }je analyzed the«^ interaction patterns in conjunction with the students'* 

' : \ ^ • • ^ ' ^. 

characteristics, it appeared th^t Interaction patterns requiring more free 
responding were mgre effective, particularly ^with studentsi^who had studied 
some English (^d again irrespective of the aoti^al lev^ of proficiency) « 

Jhe reader hap undoubtedly noticed the frequency with which the "Other" 
interaction pattern appeared significant in th^se analyses* When we examined 



the ^Variety of specific instances in this category, m^y of them seemed to 
^e(iuire the student to generate language, to think about the lang^uage (for - 
example, sorting words into their grammatical categoriesn,, and to use the 
language for reading arid^ discussion.^ This category and ^'-free response" ^ 
appear frequently as sigm^ficaht interaction patterns parCicularly in 
conjunction with students' characteristics. The activities in tll^ese catjsgories 
seem to have in common increasing the frequency with whi^h student;s,,use the^ 
language^ but use it not by imitating Ht but by generating it. . ^. ^ 



.A practical plan might be to use the variants on the audiolingual Jnetho'd 
^^^with students Vho have the least prof iciency^, but ev^n then to mix it wiLt'h ^ 
^ the other interaction pa'ttems as quickly as possible. Perhaps i^he next type 
*^ ^ interaction pattern to ulse af ter*" some minimal proficienqy has be^n acquire^ 
is the forms of the "Teacher-direct Jst.udent read and/or^^ask ^^lueationa." This" 



1 



type of- procedure, seems to be a^way of helping students to use th'e language..* 
natprally* But the direction of using thes^ procedures hould be to bring 

the students to free responding.** ^ . , 

^ ' '\'^ ' ' * 

*' It .is' clear that no onfe method or c.6aching performance is uniformly 

« / ' ' • '/V ... ^ , . 

effective. One cannot really take ^idfes in an au41olingual v6. .^'silent 

> . r ' \ < / , ^ ' ' ^ ' , 

way" deb^ate^on methodology since the^ffectiye patterns of teacher behavior . ^ 
cut across the-eiements of both. This conclusion is 'particularly important 
because the' teachers - tend to use a consistent style. The effects Of this 
consistency were apparent in the analysis tff different classes of the same - 
teacher. One class of a teacher achieved bette r than pr edicted; another, . 
taught in the same way,'4id le'ss well than predicted. Only when^we analyzed 
the interaction patterns with the student characteristics' data did we find 
tha^t^me^hods probably must b^ adapted to specific characteristics of the 

students. * . 

The regressioji analyses^ indicated that students' characteristics were a 
major predictor of their subsequent achifevement. This factor cannot *e 
ignored if one.wishes'to make the system of, inst^ruction.even more effective. 

are not implying that the Center's teachers do hot adapjt, to the t3rpe o^'x,^, 
class; some do, some haye selected the appropriate method for thfe class^ich 
they are teaching. But obviously th^ere were. classes for which the instructional 
methods were inappropriate or less effective. Qiven that thie^^||||^r's Teachers 
wdrk closely wlth^ their students^ it seems likely that giving the teachers more^ 
. informaeion about their students (such as that provided hy tlje questionnaire 

developei for this study), 'ai^ learning which procedures ^afq more effective 

, , ' * . ' ' ' , ' 

with certain types of students may be sufficient to ^increase"* the' effectiveness 

/ ' ^ ^ ... - . ^ ' ' ^ 

ofrthe fsystem inar)cedly. ' • . ' - ' ' ' " 



•Tbe reader should recall that, this study was not an evaluation of 
the Center. It was an intensive.^ ih-depth study' of the studehts and " 
teachers or..th; Center' to find out bow best to organize instruction. • 
W results have provided some hypo th^f^C^wfelch plans of action may 
be built. 
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